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(57) ABSTRACT 

A system and method for correction and reconstruction of 
digital color images make use of one or more of a set of 
algorithms for color calibration and correction, and recon- 
struction. An algorithm for optimized bit depth reduction 
also can be used to match the response curve of the scanner 
to that of the scanned media, thereby improving signal-to- 
noise ratio and decreasing artifacts such as pixelization, 
which can result from sampling the tone curve too coarsely. 
In a photographic film application, in particular, a color 
calibration and correction algorithm enables correction of 
the image for variations in hue from film type to film type, 
over-exposure or under-exposure, exposure-induced hue 
shifts, hue shifts caused by lighting effects, processing 
related hue shifts, and other variables in film processing, 
while preserving overall hue of the subject matter in the 
originally photographed image. An image reconstruction 
algorithm allows creation of look-up tables (LUTs) that 
create a visually pleasing version of the image when applied 
to the original data. 

34 Claims, 6 Drawing Sheets 
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IMAGING SYSTEM AND METHOD film, the method comprising producing average color value 

data for the scanned color image, performing exposure 
This application claims priority from U.S. Provisional correction of the image using the average color value data 
Application Ser. No. 60/092,867, filed Jul. 15, 1998, the and exposure calibration data, performing chromatic correc- 
entire content of which is incorporated herein by reference. 5 of ^ image using a subject failure suppression bound- 
ary following the exposure correction, generating image 
TECHNICAL FIELD correction data representative of the exposure correction and 

the chromatic correction, and applying the image correction 
The present invention relates to techniques for scanning data to the image to produce a corrected color image, 
and reproducing images and, more particularly, to tech- 1Q In another embodiment, the present invention provides a 
niques for processing scanned image data to generate a system for correcting a digital color image scanned from 
digital image for display or reproduction. film, the system comprising means for producing average 

color value data for the scanned color image, means for 
BACKGROUND performing exposure correction of the image using the 

Creation of positive images from photographic negatives 15 average color value data and exposure calibration data, 
can be a difficult and imprecise process. The fundamental means for performing chromatic correction of the image 
problem is that the consumer waits to receive images that a faJurc oppression boundary following the 

are consistently the correct color balance and brightness correction means for generating image ^correction 

despite variables in the photographic imaging process. dat f .representative of the exposure correction and the chro- 
Sources of variation in the photographic imaging process 20 ™tic ^correction, and means ;for applying the ^image correc- 
include: (a) variations in spectrophotometry and sensito- aon data to lhe ima S e t0 P roduce a color 
metric characteristics from film type to film type, (b) emul- In a h« h ™ embodiment, the present mvention provides a 
sion to emulsion variation within a film type, due to film method for reconstructing a digital color image scanned 
manufacturing variability, film shelf aging before exposure, from film > the method comprising producing average RGB 
latent image fading after exposure, dark fading after 25 color value data for the scanned color image, performing 
processing, and chemistry variations in the film processing exposure correction of the image using the average color 
step, (c) illumination variation at the time of photography, vahie data and exposure calibration data, performing chro- 
which can cause both color balance variation in the image matic correction of the image using a subject failure sup- 
and exposure level variation, and (d) other variations, such P ression boundary following the exposure correction, geo- 
as those arising from camera lens color differences. The 30 erating image correction data representative of the exposure 
challenge in photographic color correction is compensating correction and the chromatic correction, generating recon- 
for each of the above variables, while at the same time struction lookup tables (LUTs) based on the color correction 
preserving color deviations from neutral in the image that data and tne avera ge color value data, each of the recon- 
are caused by the subject matter that was photographed. struction LUTs representing a curve for reconstruction of 

35 one of the RGB color channels for the image, and applying 
SUMMARY each of the reconstruction LUTs independently for the 

respective RGB color channels to produce a reconstructed 
The present invention is directed to a system and method color image, 
for correction and reconstruction of color images generated In m adcJed embodiment, the present invention provides 
by a scanner. The system and method make use of one or ^ a systcm f or reconstructing a digital color image scanned 
more of a set of algorithms for color calibration and from ^ ^ system comprising means for producing 
correction, and reconstruction of scanned images. The average RGB color value data for the scanned color image, 
images can be scanned, for example, from reflective or means for performing exposure correction of the image 
transmissive film or paper. In particular, the images can be mc avcragc co j or va i uc d ata exposure calibration 

scanned from processed negative or positive photographic 4S data ^ means for performing chromatic correction of the 
film. Other examples include photothermographic or ther- a SUD ject failure suppression boundary follow- 

mographic film, elcctrographically printed paper, inkjet me exposure correction, means for generating image 

printed paper, and the like. For ease of illustration, all of the correction data representative of the exposure correction and 
above media will be referred to herein as "film." the chromatic correction, means for generating reconstruc- 

In a photographic film application, for example, a color 50 tion lookup tables (LUTs) based on the color correction data 
calibration and correction algorithm enables correction of and the average color value data, each of the reconstniction 
the image for variations in hue from film type to film type, lookup tables representing a curve for reconstruction of one 
over-exposure or under-exposure, exposure-induced hue of the RGB color channels for the image, and means for 
shifts, hue shifts caused by lighting effects, processing applying each of the reconstruction LUTs independently for 
related hue shifts, and other variables in film processing, 55 the respective RGB color channels to produce a recon- 
^ while preserving overall hue of the subject matter in the structed color image, 
originally photographed image. An image reconstruction i Q another embodiment, the present invention provides a 
algorithm allows creation of look-up tables (LUTs) that method for correcting a digital color image scanned from 
create a visually pleasing version of the image when applied uull( the method comprising producing average color value 
to the original data. If desired, the system and method also 60 data f or the scanned color image, performing exposure 
may use an algorithm for optimized bit depth reduction that correction of the image using the average color value data 
more effectively matches the response curve of the scanner anc j exposure calibration data, performing chromatic correc- 
to that of the film, thereby improving signal-to-noise ratio ti 0 n of the image using a subject failure suppression bound- 
and decreasing artifacts such as pixelization, which can arv following the exposure correction, generating image 
result from sampling the tone curve too coarsely. 65 correction data representative of the exposure correction and 

In one embodiment, the present invention provides a the chromatic correction, and applying the image correction 
method for correcting a digital color image scanned from data to the image to produce a corrected color image. 
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In a further embodiment, the present invention provides a reflective mode. In one embodiment, scanner 12 may incor- 
system for correcting a digital color image scanned from porate scanning optics such as a line scanner oriented to 
film, the system comprising means for producing average apply a beam of light to processed, i.e., developed, photo- 
color value data for the scanned color image, means for graphic film, a light detector such as a CCD line scan array 
performing exposure correction of the image using the 5 arranged to receive light transmitted through the film, and 
average color value data and exposure calibration data, appropriate color filters for acquisition of color separation 
means for performing chromatic correction of the image data from the CCD line scan array, e.g., red, green, and blue 
using a subject failure suppression boundary following the (RGB) color separation data. The light detector alternatively 
exposure correction, means for generating image correction could include separate detector elements that are sensitized 
data representative of the exposure correction and the chro- to respective color separations, eliminating the need for 
matic correction, and means for applying the image correc- color channel filters. As an example, scanner 12 can be 
tion data to the image to produce a corrected color image. configured to scan each color separation at n-bit gray level 

Other advantages, features, and embodiments of the accuracy, and then pass the resulting data through a n to 

present invention will become apparent from the following m -bit conversion lookup table (LUT) in hardware integrated 

detailed description and claims. with the scanner, wherein m is less than n. In this manner, 

DESCRIPTION OF DRAWINGS scanner 12 can perform bit depth reduction to more readily 

rrw si * . t. i.iij' r * r_ maintain a high scan throughput in terms of film frames 

FIG. 1 is a functional block diagram of a system for anne d r minute 

correction and reconstruction of scanned color images; scanne per mmu . 

FIG. 2 is a conceptual graph of a histogram generated for The architectures of some scanners do not allow the entire 

a hypothetical scanned film image; 20 n " bl1 dynamic r^ge * be used to its full potential. For 

FIG. 3 is a graph of a color correction and image ° xam P k ' a scanrer sometimes does not use the entire 

° r j • dynamic range avauable because some of the dynamic range 

reconstruction curve for a scanned image; . . j ■ t_-i-7 

^„ A . , f r _ is used to accommodate scanner-to-scanner variability 

FIG. 4 is a graph of a contrast shaping curve for apph- q ^ ma nufacturing variations and lamp brightness 

cation to a curve as shown in FIG. 3; . . „ variations caused by aging. In some cases, as much as half 

FIG. 5 is a flow diagram illustrating a scan calibration of the dynamic range of the scanatT can ^ dedicated to 

method implemented by a system as shown in FIG. 1; and accommodating such variations. 

FIG. 6 is a flow diagram illustrating a scan correction and ^ [a neither ^ exposure intensity nor 

reconstruction method implemented by a system as shown in thc exposure linie be changed from to image . ^ 

30 a result, over-exposures, normal exposures, and under- 
like reference numbers and designations in the various exposures must all be scanned within a common dynamic 
drawings indicate like elements. rangCi This limitation severely reduces thc remaining 
DETAILED DESCRIPTION dynamic range for any given image, because each image will 

FIG. 1 is a functional block diagram of a system 10 for ^ a ^, e one ex P° sure | eve J" , 
correction and reconstruction of scanned color images. As » Finally ui a photographic film i scanning application, pho- 
shown in FIG. 1, system 10 may include an image scanner tographic film does not have a linear relationship between 
12, and a software-based system 14 incorporating an expo- transmittance, which is what the CCD detector measures, 
sure curve generation module 16, a histogram generation and exposure. Instead, this relationship is logarithmic, 
module 18, a color correction/image reconstruction module Consequently, gray level resolution which is adequate (or 
20, and a LUT conversion module 22. For an exemplary «> even more than needed) in the lighter areas of the film can 
photographic film scanning application, scanner 12 may take P/ 0Vlde inadequate resolution m the .darker areas Therefore, 
the form of a photographic film image scanner of the type * «™ * 2 undertakes a simple hnear bit depth reduction 
typically used to scan negative film in roll format Exposure ^ m tbc D " blt m P ut <° th f m-tat output, ^Jight areas ofthe 
curve generation module 16 processes calibration data gen- film f rc oversampled and the dark areas undersampled. TTiis 
crated by scanner 12 in response to caUbration images, and " problem is most evident, on negative processed photo- 
produces calibration cur^s lor conec^ oroverlxposed film > m highlights of overexposures. In this case, the 
and underexposed images. The calibration curves can be undersamphng causes the measured levds to be severely 
stored for later use. Histogram generation module 18 pro- quantized, leading to pixehzation in the reconstructed 
cesses image data generated by scanner 12 in response to image. 

actual user-provided images, and produces a histogram 50 WitD & & effects described above, after the n to m bit 

representing the distribution of the gray levels for the red, conversion, typical images produced by the scanner may 

green, and blue (RGB) channels within the image. Color have a substantially reduced dynamic range. Such a dynamic 

correction/image reconstruction module 20 processes the * <l uite adequate for many applications, especially if 

histogram generated by histogram generation module 18 and dynamic range is properly used. However, individual 

calibration curves generated by exposure curve generation 55 colof separations of some images may have even fewer gray 

module 16 to produce correction LUTs for each of the red, lcvds - In the numbcr of levels ma y be to ° 

green, and blue color separations. Further, color correction/ fcw t0 reconstruct an acceptable image. Images that repre- 

image reconstruction module 20 generates LUTs represen- sent with low brightness ratio are by definition scenes 

tative of a image reconstruction curve. LUT conversion with a lower number of gray levels. Accordingly, an algo- 

module 22 applies the reconstruction LUTs to the actual 60 rithm for optimized bit depth reduction can be implemented 

image data generated by scanner 12 to produce a recon- within scanner 12 whereby the gray levels available in the 10 

\ structed image that can be used to generate high-quality bit image are used more wisely to minimize artifacts. The 

Reproductions of the original imagery. algorithm can be implemented in hardware, firmware, soft- 

ware or a combination thereof within scanner 12. 

Optimized Bit Depth Reduction 6S Alternatively, optimization of bit depth reduction could be 

Scanner 12 may conform substantially to conventional assigned to a device external to scanner 12 that receives the 

image scanners useful in scanning film in a transmissive or n-bit, e.g., 10-bit, output from the scanner. As a further 
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alternative, scanner 12 may produce n-bit data that is sub- such as a conventional keyboard and pointing device such as 

jected to optimized bit depth reduction via a software a mouse or trackball, and an output device, such as a CRT 

module associated with system 10. In particular, optimized or flat panel computer monitor that provides textual and/or 

bit depth reduction can be integrated with the color correc- graphic output to the user. 

tion algorithm and carried out by color correction/image 5 The color correction algorithm can be applied, for 

reconstruction module 20. In this manner, optimized bit example, to digitized output data generated according to a 

depth reduction may form part of the color correction. optimized bit depth reduction algorithm, as described above. 

The inherent transmittance of photographic film is loga- Alternatively, the color correction algorithm can be applied 

rithmic. Therefore, it makes sense for the detector measuring to output data that has not been subject to bit depth reduc- 

transmittance to measure logarithmically as well. The tone 10 tion. 

curve optimization algorithm takes this approach, and Algorithms for achieving accurate color correction have 

thereby reduces the complexity of the conversion. Accord- been developed for conventional photographic printing 

ing to the tone curve optimization algorithm, the values of equipment. The so-called "integration to gray" theory, for 

the darkest usable portion and the lightest exposed portion of example, is disclosed in U.S. Pat. No. 2,571,697 to Evans, 

the film are measured in the 10-bit space. The range is then 15 a color correction algorithm based on the "integration to 

scaled using a gamma correction curve, y-ax*+b, instead of gray" theory can be implemented using a transformation 

a logarithmic curve, mapping the range to a 2 m -level space. fr 0 m red-green-blue color space to a hue, saturation, and 

The 2"Mevel space is mapped, however, subject to the lightness (HSL) space. A color correction algorithm in 

constraint that the maximum step size is 1. This constraint accordance with this embodiment may use the "integration 

forces the algorithm to use every available m-bit value, 20 to gray" approach. This algorithm is designed to work in any 

keeping the measurement as smooth as possible. The use of 0 f a number of HSL color spaces. 

a gamma correction type curve instead of a logarithmic HSL color spaces can be expressed as either cylindrical 

curve is preferred because the gamma curve is easier to coord i natc systems or Cartesian coordinate systems, and 

handle from a computational standpoint. Also, from a prac- generaUy have mc following characteristics: (a) when the 

tical standpoint, the two curves have a very similar shape, so ™ colof fa ^ as cylindrical coordinate systems, 

that in actual use, the results are similar with either curve. variations aloag the mela axis are perceived by humans as 

An advantage of this technique is that it better utilizes the shifts m the hue (e.g. red, yellow, green, cyan, blue, or 

inherent n-bit accuracy in that portion of the transmittance magenta), variations along the radial axis are perceived as 

curve of the film where it is most needed, i.e., in lighter shifts in the saturation, e.g., low saturations are grayish, high 

areas, while giving up some accuracy in portions of the saturations are intense or colorful, and variations along the 

transmittance curve that are being oversampled. In other z-axis are perceived as changes in the lightness of the object; 

words, accuracy is improved for the lighter portions of the anc ) (b) when the color space is expressed as Cartesian 

processed photographic film and reduced in the darker coordinate systems, the x and y axes correspond to two 

portions. Thus, there is gain in accuracy where it is needed opposite color pairs (frequently one is the red/green axis, 

without appreciably sacrificing accuracy in any other portion an d the other is the yellow/blue axis), and the z axis is again 

of the film. Utilizing this technique significantly decreases a lightness axis. 

the level— both frequency of occurrence and severity— of j ^ 0Qe of well HSL color spaces . 

pixelization artifacts. A T space ^ ^5^^ f or example, in U.S. Pat. No. 

„ . _ ... . . - . 4Q 4,159,174 to Rising and U.S. Pat. No. 4,154,523 to Rising et 

Color Calibration and Correction t ^ • 1 « r<t 1 1 *uJr- 

al. For implementation of the color correction algorithm m 

In another embodiment, the present invention implements a photographic film application, T space may be desirable 

an algorithm for color correction of digitized imagery, such due to both its familiarity among film scanner users, and its 

as that scanned from processed photographic film, or other ready correlation with the physical output of the film, 

film or paper media. Color correction can be implemented in 45 Specifically, the x and y axes of T space are the GM 

software, in part by exposure curve generation module 16, (green/magenta) and ST (skylight/tungsten) axes. White 

which generates calibration exposure curves for correction objects illuminated by mixtures of outdoor lighting and 

of variations due to under-exposure or over-exposure of a indoor (incandescent) lighting appear at differing places 

scanned image, histogram generation module 18, which along the S-T axis depending on the percentage of light from 

generates histogram information representative of density 50 each source. This fact is useful in understanding the color 

levels for a given color separation, and color correction/ correction algorithm described herein. Application of the 

image reconstruction module 20, which generates correc- algorithm in T space will be described for purposes of 

tions based on the histogram information and the calibration example. However, other possible HSL spaces would work 

exposure curves. This software implementation can be just as well. Also, the conversion to the chosen space can be 

arranged in program code that is accessed and executed by 55 isolated to one small section of the computer code used to 

a processor. implement the algorithm, thereby facilitating ready change 

The program code can be loaded into memory from if use of another space is desired. A conversion to another 

another memory device, such as a fixed hard drive or «>>«■ sp ace could require, however, recalibration of the 

removable media device associated with system 10. The system and reflection of the subject failure suppression 

processor may take the form of any conventional general 60 boundaries (SFSB or < 'woodpecker boundaries") and recon- 

purpose single- or multi-chip microprocessor such as a struction targets, as will be described. 

Pentium® processor, a Pentium Pro® processor, an 8051 After the images have been scanned and output data has 

processor, a MIPS processor, a Power PC® processor, or an been produced, e.g., using an algorithm for optimized bit 

Alpha® processor. In addition, the processor may be any depth reduction within scanner 12 as described above, the 

conventional special purpose microprocessor. Further, the 65 color correction algorithm is applied to correct exposure 

processor can be integrated within a personal computer or level and remove unwanted color casts. The color correction 

computer workstation that incorporates an input device, algorithm is scanner independent. The first step of the 
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algorithm is to generate a histogram of the number of pixels the image to the lightness value of the standards for that DX 

at each brightness level for each color separation. The code. The Lightness correction is set equal to the difference 

histogram information is used, in part, to yield average color between the image and the normal exposure standard. Color 

value data indicating the average color values within the corrections along each of the color axes are calculated as a 

image. Other conventional methods for producing average 5 fraction of the color errors of the over- or underexposure 

color value data can be used. With reference to FIG. 1, the stan dard, depending on whether the image was determined 

generation of average color value data can be assigned to to be over . or un d ereX posed. The fraction chosen is the ratio 

discrete histogram generation software module 18 within of me dis , ance betmm the fa ^ the QOrmal stfmdard 

software system 14. The resulting histogram information for to (he betweeQ ^ over . Qr standard . 

each color separation is passed into the color correction 1Q ^ amounts to assumin ^ ^ e e mduced hue 

routine tmplememed w.thin color correcuonAmage recon- shift fe ^ wim ure ^ „ , his fe not exactl 

stmction module 20, as shown m FIG. 1 It is notable that the acmal shifts m c f ose eD0U ^ tQ f inear „, in ti<x> 

histogram informauon is Uie only mformation about the ^ tion ^ ite In other embodiments, the 

color content of the scanned image that the colorwrrectionV hu6 ^ versus e e ^ ^ be modeled ^ 

image reconstruction module 20 need receive. This feature „ linc> ^ lynonjial; or otber ^.fitting functions, 

is advantageous for commercial settings in which scanner 12 m ...... , 

is used to scan images from film having diverse origins and V 16 ** ^rectwns are stored by the algorithm and applied 

characteristics, such as film received from amateur photog- 10 ^ "™B* ^mates. After this step there is a 

raphers in a photo processing shop. »« °. £Ts P ac « ~ordmates having a lightness of 0.0, meaning 

»# j i in i i . .l nrT11 i • • me image has the same exposure as a properly exposed 

Module 20 calculates the average RGB level in the image ~ ft 4 . ° ™ ^ _j- * f \. o £ j 

, r , . , . . c . 4 . if _ , r 20 neutral scene. The T space coordinates also have S-T and 

by reference to the histogram mformation, takes the log of ~ 1/ 1 ^ * f. - 1t , 

.11* . r *u * .u *t j G-M values that are typically non-zero, and represent a 

such levels to account for the gamma of the film, and . - . , jr , , J , ' r . 

4 iL , 4 „ J 5 1 . j » mixture of lighting mduced hue shifts, other objectionable 

converts the values to T space. T space is related to RGB , j • c *u *, 1 1 i *u 

, . * ■ n- i- t> . . hue shifts, and variations of the actual color of the scene 

space by a simple matnx mu tiphcation. By examining the from has e 

location (relative to a normal exposure of a standard gray j- * r *i_ • 

x c \, j » » ■ . ™ « 25 exposure-corrected T space coordinates of the image, nota- 

scene) of the average coordinates of the scene m T space, the , , . . , r ^ & 

7 . . r °, , » . j * 1 bly prior to chromatic correction. 

exposure level of the scene can be estimated. Also, any J r 

systematic color shifts caused by lighting or other external ™ e exposure correction will introduce a desirable color 

influences can be determined from the S-T and G-M coor- smft m the corrected image if the over- or under-exposed 

dinates of the scene. Because the comparison is made to a 30 ^S* 5 is not ncutral relative to the normai exposure. This 

standard of the same film type, film type-to-film type varia- situation is common and is known as exposure dependent 

tions can be eliminated by calibration. For further informa- color If me chromatic correction (using the subject 

tion concerning film type calibration, reference is made to E. suppression failure, or "woodpecker," boundary) were based 

Goll, D. Hill, and W. Severin, Journal of Applied Photo- on lhe exposure-uncorrected coordinates, the gray point 

graphic Engineering, Vol. 5, Num. 2, Spring 1979, pp. would 001 trul y g rav - Instead > tbe S ra y P 0 * 01 be 

93-104 away from gray by however far the exposure correction 

In order to carry out the color correction, standard refer- c *™&? the S " T ™ d G '™ ^ ues f ™ e color correction 
ence films are obtained for each film type that is scanned. a *g°^ avoids this result by performing the chromatic 
Because over- and under-exposures undergo hue shifts as correction after the exposure correction, 
well as changes in lightness, it is necessary to calibrate 40 For gray correction, step (2), further color corrections are 
against such shifts. Thus, the calibration film should consist generated, but not yet applied, by calculating the difference 
of a normal exposure, an underexposure, and an overexpo- between the normal exposure standard and the S-T and G-M 
sure of a reference scene that is, on average, neutral gray. An values of the exposure-corrected scene. If applied, these 
example of suitable calibration films are the True Balance corrections would make the average hue of the corrected 
calibration strips made by Aperion Company for calibrating 45 irn a S c neutral gray. However, not all scenes are, on average, 
conventional analog photoprinters. Other calibration films neutral gray. Accordingly, the gray correction data is held in 
that satisfy the above criteria would suffice. For calibration, abeyance for performance of the chromatic correction, 
each image is scanned, averaged, the logs are taken, and For chromatic correction using the subject failure sup- 
converted to T space, just as for the images to be corrected. pression boundary, step (3), a comparison is made between 
This data is stored for each film type under a file name for 50 the size of the preliminary color corrections previously 
its unique film type identifier code, i.e., a DX code. DX determined from step (2) and a boundary in T space known 
codes for photographic film are discussed in detail in Ameri- as the subject failure suppression boundary, and colloquially 
can National Standards Institute standard ANSI/NAPM known by those skilled in the art as the "woodpecker^* 
HI. 14-1 994. In this manner, calibration data is assembled boundary because of its resemblance to a profile view of the 
for a variety of film manufacturers, film types, and film 55 head of a woodpecker. This boundary is predetermined by 
speeds. Many conventional film scanners include a bar code examining the T space coordinates of many images, on an 
reader configured to read the DX code from a roll of film and empirical basis, and hand picking a boundary which will 
pass it to processing software along with the scanned image provide appropriate correction. A line is projected from the 
data. origin of T space, through the coordinates of the preliminary 

After image coordinates in T space have been calculated, 60 co * or corrections of step (2), and to the point where the line 

the corrections required to move the image back to normal intersects with the boundary. 

exposure and remove color casts are calculated in four steps: A fraction is generated by dividing the distance of the 

(1) exposure correction, (2) gray correction, (3) chromatic correction from the origin of T space by the distance of the 

correction using tbe subject failure suppression (or boundary from the origin at the point of intersection. If the 

"woodpecker,") boundary, and (4) final correction. 65 fraction is below some lower limit (frequently 0.2), it is 

For exposure correction, step (1), image over- or under- adjusted up to that lower limit. Then, the preliminary color 

exposure is corrected by comparing the lightness value of corrections from step (2) are multiplied by the fraction to 
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generate a new set of corrections. The purpose of this step intended display device, if known. Because of compromises 

is to remove the effects of lighting-induced hue shifts while with other image appearance factors, a neutral gray in the 

not removing scene content-induced hue shift. When the final image may not actually be mapped to this color, 

exact position of the boundary is chosen, it is carefully Following determination of the target color, manual 

selected with this goal in mind. Implementation of the color 5 adjustments may be made, as will be described in greater 

correction algorithm as described herein allows the number detail later in this description. Gamma then is determined by 

of points used to define the boundary to be varied easily calculating the parameters required to map the midpoint to 

depending on the characteristics of the film, application, and me target color, while mapping the lower and upper ends of 

the expected subject. Also, the algorithm allows the easy use me histogram to 0 and 255, respectively. Mathematically, a, 

of a different boundary for each film type. 10 D , and g are calculated for the following equation: y=a x 5 +b. 

For the final calculation of corrections, step (4), the actual Stretch of the gamma correction curve can be limited, if 

corrections to be applied to the scene are calculated by desired, as will be described in greater detail later in this 

adding together the exposure corrections generated in step 1 description. A nearly final ("intermediate") version of the 

and the gray level/chromatic corrections generated in step 3. LUT is calculated next, using the gamma correction equa- 

These corrections are converted back to RGB space, where 15 tion. An example of the curve represented by the LUT is 

they represent relative shifts in the scaling of the RGB data. shown in FIG. 3. 

These shifts, and the original histograms representing the contrast of the final image can be adjusted by 

image, are passed onto the image reconstruction algorithm. applying a contrast shaping curve, as shown in FIG. 4, as a 

The image reconstruction algorithm may conform substan- separate look-up function, to the intermediate LUT. This 

tialiy to that described below, or may be selected from other 20 s t ep both suppresses overstretch and lends a more aestheti- 

available algorithms. Accordingly, correction and ca u y pleasing appearance to the image. As shown in FIG. 4, 

calibration, in accordance with the present invention, can be the rounded sections at the ends of the shaping curve tend to 

practiced separately from image reconstruction, as described suppress pixelization, which typically occurs in the brightest 

herein, and vice versa. or darkest regions of the image, and the higher slope in the 

25 center increases midtone contrast. The purpose of the con- 
Image Reconstruction mst shaping function is to modify the nearly final LUT to 
An image reconstruction algorithm can be implemented in achicve aesthetically pleasing results in the reconstructed 
software by color correction/image reconstruction module Man y possible functions could be used for the 
20. For image reconstruction, the only information available 30 contrast shaping function. Included below is a discussion of 
is the R, G, and B histogram information for the scanned an exemplary contrast shaping function. The contrast shap- 
image. Even after the color corrections have been applied, ^g function, in this example, can be represented by the 
only one color — the average hue of the image — has been following equation: 
corrected. Moreover, it is not even known what calorimetric 
values that color had in the original scene of the scanned 



^(l-x^i tan h(m x (x-0S))+xs 2 Un h(m^x-0S))yi 



image. TTiere is no known white or black point, because no In ^ abovc e^o^ tarm represents the mathematical 

relationships between the R, G, and B values of any given function known as a hyperbolic tangent. The function is used 

pixel are known. The relationships were lost when the 5y the ncarly ^ output value of cach channel to 

histograms were made. Similarly, there is no information to me range bfltween 0 and 1 by dividing it by the value of the 

facilitate location of key colors like flesh tones, foliage w largest a]Jowed 011tput va lue. In an 8-bit system, for 

green, sky blue, or neutral gray. This lack of colorimetric cxamp \^ the largest allowed output value is 255. The scaled 

information means that the image reconstruction algorithm value ^ then inserted as x into the contrast shaping function, 

must be largely ad hoc. It can only be based on a general ^ y vahie * scaled back to the range between 0 

knowledge of film behavior, knowledge of how the human and me largest allowe d output value by multiplying it by the 

visual system works, and assumptions about likely scene 45 hrgest alloW ed output value. 

content Therefore, the image reconstruction algorithm is The constants &1 and ^ in the contrast shaping function 

highly heuristic, but has been observed to produce good arc ^ 0SCQ to kccp mc output valuc between 0 and 1. 

image quality results despite the dearth of information Typically, constants Sj and ^ are chosen so that the lowest 

concerning the original image. possible output value is 0 and the highest is 1. However, in 

According to this image reconstruction algorithm, each 50 some of the low contrast cases detected in the" limit stretch" 
channel (R, G, and B) is treated independently, and each step step, they may be chosen so that the lowest possible output 
below is applied separately to each channel. First, a pre- value is greater than 0 and/or the highest possible output 
liminary reconstruction LUT is calculated. The preliminary value is less than 1. The constants m 2 and m 2 are chosen to 
reconstruction LUT represents a "first estimate" of the control the slope of the curve, and therefore the contrast of 
reconstruction LUT, and is calculated by applying the scale 55 the image, in the middle of the curve (corresponding to the 
factors calculated in the color correction algorithm to the midtones of the image) and the amount of curvature in the 
integers between 0 and 255. Second, the algorithm locates high and low values of the curve (corresponding to the 
extremes and midpoints. In particular, the lower and upper highlights and shadows of the image). Increasing the cur- 
points of the histogram which actually contain data are vature of the curve can cause it to have a lower slope, and 
located, as is the (uncorrected) average point. The prelimi- 60 hence lower contrast, in selected areas of the image. This can 
nary reconstructed points corresponding to these three posi- be useful for, among other things, minimizing the effects of 
tions are also located, as illustrated by FIG. 2. pixelization artifacts. In the example curve shown in FIG. 4, 

Next, a target color is determined. Specifically, the target Sj-s^l/ptan 6(0.5)) and m 1 =m 2 =l. It is not necessary that 

for the average color to which the image is to be mapped is Si equal S 2 , or that M x equal M 2 ; they can be chosen 

loaded from a file. This target color is conceptually a neutral 65 separately. S A and M x primarily affect the shape of the curve 

gray. However, it may be chosen somewhat away from gray for low x values, whereas and M 2 primarily affect the 

to accommodate the color characteristics of the final shape of the curve for high x values. 
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As a final step, if the film is negative film, the LUT is 
inverted so that it will create a positive image when applied 
to the image data. LUT conversion module 22 then applies 
the final correction LUT to the scanned image data to 
reconstruct the scanned image for reproduction. The recon- 
structed image can be stored as an image file for later use, 
or immediately printed as a positive image on a conventional 
film printing device. 

The basic algorithm described above, without the manual 
adjustment and stretch limit steps, has been observed to 
provide good results in most instances, and especially with 
"well behaved'* images, i.e. those containing more than 100 
gray levels per separation and having histograms that are 
either approximately flat in the areas where there are data, or 
peaked toward the middle. However, it is sometimes desir- 
able to manually adjust overall scene brightness, contrast, 
and/or color balance. For this reason, color correction/image 
reconstruction module 20 can be configured to allow input 
of such adjustments. Thus, in the manual adjustment step, 
brightness and color balance adjustments can be made by 
adjusting the color target for each separation up or down 
depending on the direction and amount of change requested. 
Changing brightness involves adjusting all the channels 
equally, whereas changing color balance is achieved by 
adjusting them individually. Changing image contrast can be 
achieved by increasing or decreasing the slope of the con- 
trast shaping curve. 

As mentioned above, pixelization and quantization arti- 
facts sometimes show up in images with relatively few gray 
levels in each separation. Because the reconstruction algo- 
rithm uses a non-linear mapping between input and output 
(this is done to reflect the fact that film response to exposure 
is non-linear; and in fact has substantially the same shape as 
the gamma correction curve), some gray levels end up with 
larger gaps between them and their neighbors than they 
would have in a linear reconstruction, while some end up 
with smaller gaps. Unfortunately, this gap variation can 
increase the artifacts in the large gap areas of the tone curve. 
This problem can be referred to as overstretch, because it 
results from stretching the dynamic range of the image too 
far in the attempt to make it fill the dynamic range of the 
output device. Overstretch can be detected and corrected io 
at least two ways. 

First, if the overall dynamic range of the input image is 
below a given threshold, the contrast shaping curve can be 
adjusted to lower the contrast of the reconstructed image. 
This operation causes the reconstruction of a dynamically 
flat scene such as, say, an overcast sky, to be reconstructed 
as a gray image with subtle color and brightness variations, 
which is what it actually looks like. This is in contrast to 
having wildly modulated brightness shifts, which the basic 
algorithm could try to introduce by stretching the histogram 
to fill the whole range from 0 to 255. Second, images with 
histograms that are highly skewed to one end or the other 
can be identified. An example is a picture of campers 
gathered around a campfire at night. The image is mostly 
black, but has some light pixels corresponding to the fire. 
Consequently, this image exhibits good dynamic range and 
would not be detected by the dynamic range test described 
above. When the reconstruction stretches the average bright- 
ness point, which was in the middle of the hump of nearly 
black pixels, to the middle of the dynamic range, all the 
pixels darker than the average are overstretched. This case 
can be detected by directly looking for cases where the slope 
of the reconstruction curve is too high, and adjusting the 
brightness appropriately to bring it down. 

FIG. 5 is a flow diagram further illustrating the operation 
of a scan calibration method implemented as described 



.2,835 Bl 

12 

above. With reference to FIG. 5, a non-linear 10 to 8-bit 
LUT is first generated to permit optimization of the tone 
curve produced by scanner 12 following bit depth reduction, 
as indicated by bbek 24. Again, the non-linear LUT can be 

5 constructed as a gamma correction curve to better match the 
logarithmic response of the film. Also, a maximum step 
constraint can be imposed to ensure that the entire 255 value 
range is utilized. With the non-linear LUT pre-calculated, a 
calibration routine is undertaken. 

10 Specifically, as indicated by block 26, an underexposed 
calibration image is scanned by scanner 12. The 10-bit scan 
of the underexposed image is then converted to an 8-bit 
image using the pre-calculated LUT, as indicated by block 
28. Histogram generation module 18 next generates histo- 

15 gram information for each color separation, as indicated by 
block 30, and saves the average red, green, and blue values 
for the underexposed calibration image, as indicated by 
block 32. The saved average will be used in the image 
correction algorithm. 

20 Next, a normal exposure image is scanned by scanner 12, 
as indicated by block 34. The 10-bit scan of the normal- 
exposure image is converted to an 8-bit image using the 
pre-calculated LUT, as indicated by block 36. Histogram 
generation module 18 next generates histogram information 

25 for each color separation of the normal-exposure image, as 
indicated by block 38. The average red, green, and blue 
(RGB) values are saved for the normal exposure calibration 
image, as indicated by block 40. 

Next, an overexposed calibration image is scanned, as 

30 indicated by block 42, and converted to an 8-bit image, as 
indicated by block 44. Following generation of histograms, 
as indicated by block 46, the average red, green, and blue 
values are saved, as indicated by block 48. 

The calibration information is then parameterized for use 

35 in the color correction algorithm. Specifically, the average 
red, green, and blue values for the normal exposure calibra- 
tion image are stored to a calibration file, as indicated by 
block 50. Also, the inverse slopes and intercepts of the line 
connecting the overexposure curve to the normal exposure 

40 curve and the line connecting the underexposure curve to the 
normal exposure curve are calculated, as indicated by blocks 
52 and 54, for use in the color correction algorithm. 

FIG. 6 is a flow diagram illustrating a scan correction and 
reconstruction method implemented as described above. As 

45 shown in FIG. 6, the non-linear conversion LUT is 
generated, as indicated by block 56. An image is then 
scanned and converted according to the LUT, as indicated by 
blocks 58 and 60. The converted image is processed by 
histogram generation module 18 to produce histogram infor- 

50 mauon for the red, green, and blue channels, as indicated by 
block 62. Color correction/image reconstruction module 20 
then calculates T-space coordinates for the image, as indi- 
cated by block 64. Using the calibration file generated as 
described with reference to FIG. 5, the image is corrected for 

55 over-exposure or under-exposure, as indicated by block 66. 
Next, gray level correction is carried out, followed by 
chromatic correction, as indicated by blocks 68, 70, which 
pertain to gray level and subject failure suppression bound- 
ary (SFSB) correction, respectively. Calculation of final 

60 color correction data is undertaken, as indicated by block 72, 
by reference to the exposure correction and the gray level/ 
chromatic correction. 

The final correction is then used to form preliminary 
image reconstruction LUTs, as indicated by block 74. Fol- 

65 lowing location of the extremes and midpoint of the pre- 
liminary reconstruction curve, as indicated by block 76, a 
target color for the particular film is read from memory or 
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input by the user, as indicated by block 78. Optionally, a user 
then applies manual adjustments to the target color, as 
indicated by block 80, before a gamma curve for image 
reconstruction is calculated, as indicated by block 82. 

If necessary, stretch is limited to reduce artifacts, as 
indicated by block 84. Then, an intermediate reconstruction 
LUT is calculated, as indicated by block 86. Following 
shaping and inversion of the reconstruction LUT, as indi- 
cated by block 88, the resulting LUT is applied to the image 
data to reconstruct the image, as indicated by block 90. 

The present invention has been described primarily in the 
context of scanning negative color film. The algorithms 
implemented in accordance with the present invention may 
find ready application, however, in other film scanning 
systems. For example, black and white negatives could be 
handled by adapting the image reconstruction algorithm to 
handle single channel images. This can be accomplished by 
making all the channels the same, calibrating as usual, and 
scanning as usual. Further, the algorithms can be adapted to 
handle scanned positive (slide) film with some modification. 
The foregoing detailed description has been provided for a 
better understanding of the invention and is for exemplary 
purposes only. Modifications may be apparent to those 
skilled in the art, however, without deviating from the spirit 
and scope of the appended claims. 

What is claimed is: 

1. A method for correcting a digital color image scanned 
from film, the method comprising: 

producing average color value data for the scanned color 
image; 

performing exposure correction of the image using the 
average color value data and exposure calibration data; 

performing chromatic correction of the image using a 
subject failure suppression boundary following the 
exposure correction; 

generating image correction data representative of the 
exposure correction and the chromatic correction; and 

applying the image correction data to the image to pro- 
duce a corrected color image. 

2. The method of claim 1, further comprising: 
producing the average color value data by: 

producing histogram information representative of a 
distribution of RGB color values within the scanned 
color image, and 
determining average RGB color values within the 
image based on the histogram information; 
converting the average RGB color values to HSL coor- 
dinate values; and 
performing the exposure correction of the image using the 
HSL coordinate values and the exposure calibration 
data. 

3. The method of claim 2, further comprising selecting the 
exposure calibration data based on the HSL coordinate 
values and a type of the film from which the color image was 
scanned. 

4. The method of claim 3, further comprising selecting the 
exposure calibration data by: 

selecting a set of the exposure calibration data corre- 
sponding to the type of film from which the color image 
was scanned; 

determining whether the image was over-exposed, under- 
exposed, or normally exposed by comparing the HSL 
coordinate values to reference values; and 

selecting a subset of the exposure calibration data based 
on whether the image was over-exposed, under- 
exposed, or normally exposed. 
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5. The method of claim 4, further comprising performing 
the exposure correction by: 

adjusting the HSL lightness values for the image relative 
to HSL lightness values specified by the exposure 
calibration data; and 

adjusting HSL color values for the image relative to HSL 
color values specified by the exposure calibration data. 

6. The method of claim 2, wherein the exposure calibra- 
tion data includes multiple sets of exposure calibration data, 
each of the sets corresponding to a type of film from which 
the image was scanned and an exposure level of the image. 

7. The method of claim 2, further comprising scanning the 
image such that each of the RGB color values has a color 
resolution of n bits, and reducing the color resolution of the 
RGB color values to m bits following the application of the 
image correction data to the image to produce the corrected 
color image. 

8. A system for correcting a digital color image scanned 
from film, the system comprising: 

means for producing average color value data for the 
scanned color image; 

means for performing exposure correction of the image 
using the average color value data and exposure cali- 
bration data; 

means for performing chromatic correction of the image 
using a subject failure suppression boundary following 
the exposure correction; 

means for generating image correction data representative 
of the exposure correction and the chromatic correc- 
tion; and 

means for applying the image correction data to the image 
to produce a corrected color image. 

9. The system of claim 8, wherein the means for produc- 
ing the average color value data produces histogram infor- 
mation representative of a distribution of RGB color values 
within the scanned color image, and determines average 
RGB color values within the image based on the histogram 
information, the system further comprising means for con- 
verting the average RGB color values to HSL coordinate 
values, wherein the means for performing exposure correc- 
tion of the image uses the HSL coordinate values and the 
exposure calibration data. 

10. The system of claim 9, further comprising means for 
selecting the exposure calibration data based on the HSL 
coordinate values and a type of the film from which the color 
image was scanned. 

11. The system of claim 10, wherein the means for 
selecting the exposure calibration data further includes: 

means for selecting a set of the exposure calibration data 
corresponding to the type of film from which the color 
image was scanned; 

means for determining whether the image was over- 
exposed, under-exposed, or normally exposed by com- 
paring the HSL coordinate values to reference values; 
and 

means for selecting a subset of the exposure calibration 
data based on whether the image was over-exposed, 
under-exposed, or normally exposed. 

12. The system of claim 11, wherein the means for 
performing the exposure correction further includes: 

means for adjusting the HSL lightness values for the 

image relative to HSL lightness values specified by the 

exposure calibration data; and 
means for adjusting HSL color values for the image 

relative to HSL color values specified by the exposure 

calibration data. 
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13. The system of claim 9, wherein the exposure calibra- 
tion data includes multiple sets of exposure calibration data, 
each of the sets corresponding to a type of film from which 
the image was scanned and an exposure level of the image. 

14. The system of claim 9, further comprising means for 
scanning the image such that each of the RGB color values 
has a color resolution of n bits, and means for reducing the 
color resolution of the RGB color values to m bits following 
the application of the image correction data to the image to 
produce the corrected color image. 

15. A method for reconstructing a digital color image 
scanned from film, the method comprising: 

producing average RGB color value data for the scanned 
color image; 

performing exposure correction of the image using the 

average color value data and exposure calibration data; 
performing chromatic correction of the image using a 

subject failure suppression boundary following the 

exposure correction; 
generating image correction data representative of the 

exposure correction and the chromatic correction; 
generating reconstruction lookup tables (LUTs) based on 

the color correction data and the average color value 

data, each of the reconstruction LUTs representing a 

curve for reconstruction of one of the RGB color 

channels for the image; and 
applying each of the reconstruction LUTs independently 

for the respective RGB color channels to produce a 

reconstructed color image. 

16. The method of claim 15, further comprising: 
producing the average RGB color value data by: 

histogram information representative of a distribution 
of RGB color values within the scanned color image, 
and 

determining average RGB color values within the 
image based on the histogram information; 

converting the average RGB color values to HSL coor- 
dinate values; and 

performing the exposure correction of the image using the 
HSL coordinate values and the exposure calibration 
data. 

17. The method of claim 16, wherein each of the recon- 
struction LUTs is a preliminary reconstruction LUT, and 
applying the reconstruction LUTs includes: 

adjusting each of the preliminary reconstruction LUTs by 
gamma correction based on the minima, maxima, and 
midpoint of the reconstruction curve; 

applying a shaping function to each of the adjusted 
preliminary reconstruction LUTs to thereby generate 
respective final reconstruction LUTs; and 

applying the final reconstruction LUTs to produce a 
reconstructed color image. 

18. The method of claim 16, wherein each of the recon- 
struction LUT is a preliminary reconstruction LUT, and 
applying the reconstruction LUTs includes: 

selecting a target color value; 

adjusting each of the preliminary reconstruction LUTs to 
map the average RGB color value to the target color 
value and thereby generate respective final reconstruc- 
tion LUTs; and 

applying the final reconstruction LUTs to produce a 
reconstructed color image. 

19. The method of claim 16, further comprising selecting 
the exposure calibration data based on the HSL coordinate 
values and a type of the film from which the color image was 
scanned. 
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20. The method of claim 19, further comprising selecting 
the exposure calibration data by: 

selecting a set of the exposure calibration data corre- 
sponding to the type of film from which the color image 
5 was scanned; 

determining whether the image was over-exposed, under- 
exposed, or normally exposed by comparing the HSL 
coordinate values to reference values; and 
selecting a subset of the exposure calibration data based 
10 on whether the image was over-exposed, under- 
exposed, or normally exposed. 

21. The method of claim 20, further comprising perform- 
ing the exposure correction by: 

adjusting the HSL lightness values for the image relative 
15 to HSL lightness values specified by the exposure 

calibration data; and 
adjusting HSL color values for the image relative to HSL 

color values specified by the exposure calibration data. 

22. The method of claim 16, wherein the exposure cali- 
bration data includes multiple sets of exposure calibration 
data, each of the sets corresponding to a type of film from 
which the image was scanned and an exposure level of the 
image. 

23. The method of claim 16, further comprising scanning 
the image such that each of the RGB color values has a color 
resolution of n bits, and reducing the color resolution of the 
RGB color values to m bits following the application of the 
image correction data to the image to produce the corrected 
color image. 

24. A system for reconstructing a digital color image 
scanned from film, the system comprising: 

means for producing average RGB color value data for 

the scanned color image; 
means for performing exposure correction of the image 
using the average color value data and exposure cali- 
bration data; 

means for performing chromatic correction of the image 
using a subject failure suppression boundary following 
the exposure correction; 
means for generating image correction data representative 
of the exposure correction and the chromatic correc- 
tion; 

means for generating reconstruction lookup tables (LUTs) 
based on the color correction data and the average color 
value data, each of the reconstruction lookup tables 
representing a curve for reconstruction of one of the 
RGB color channels for the image; and 
means for applying each of the reconstruction LUTs 
independently for the respective RGB color channels to 
produce a reconstructed color image. 

25. The system of claim 24, wherein the means for 
producing the average color value data produces histogram 
information representative of a distribution of RGB color 

55 values within the scanned color image, and determines 
average RGB color values within the image based on the 
histogram information, the system further comprising means 
for converting the average RGB color values to HSL coor- 
dinate values, wherein the means for performing the expo- 

60 sure correction of the image uses the HSL coordinate values 
and the exposure calibration data. 

26. The system of claim 25, wherein each of the recon- 
struction LUTs is a preliminary reconstruction LUT, and the 
means for applying the reconstruction LUTs includes: 

65 means for adjusting each of the preliminary reconstruc- 
tion LUTs by gamma correction based on the minima, 
maxima, and midpoint of the reconstruction curve; 



40 
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means for applying a shaping function to each of the 
adjusted preliminary reconstruction LUTs to thereby 
generate respective final reconstruction LUTs; and 

means for applying the final reconstruction LUTs to 
produce a reconstructed color image. 

27. The system of claim 25, wherein each of the recon- 
struction LUTs is a preliminary reconstruction LUT, and the 
means for applying the reconstruction LUTs includes: 

means for selecting a target color value; 

means for adjusting each of the preliminary reconstruc- 
tion LUTs to map the average RGB color value to the 
target color value and thereby generate respective final 
reconstruction LUTs; and 

means for applying the final reconstruction LUTs to 
produce a reconstructed color image. 

28. The system of claim 25, further comprising means for 
selecting the exposure calibration data based on the HSL 
coordinate values and a type of the film from which the color 
image was scanned. 

29. The system of claim 25, wherein the means for 
selecting the exposure calibration data includes: 

means for selecting a set of the exposure calibration data 
corresponding to the type of film from which the color 
image was scanned; 

means for determining whether the image was over- 
exposed, under-exposed, or normally exposed by com- 
paring the HSL coordinate values to reference values; 
and 

means for selecting a subset of the exposure calibration 
data based on whether the image was over-exposed, 
under-exposed, or normally exposed. 

30. The system of claim 29, wherein the means for 
performing the exposure correction includes: 

means for adjusting the HSL lightness values for the 

image relative to HSL lightness values specified by the 

exposure calibration data; and 
means for adjusting HSL color values for the image 

relative to HSL color values specified by the exposure 

calibration data. 

31. The system of claim 25, wherein the exposure cali- 
bration data includes multiple sets of exposure calibration 
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data, each of the sets corresponding to a type of film from 
which the image was scanned and an exposure level of the 
image. 

32. The system of claim 25, further comprising means for 
5 scanning the image such that each of the RGB color values 
has a color resolution of n bits, and means for reducing the 
color resolution of the RGB color values to m bits following 
the application of the image correction data to the image to 
produce the corrected color image. 
10 33. A method for correcting a digital color image scanned 
from film, the method comprising: 
producing average color value data for the scanned color 
image; 

is performing exposure correction of the image using the 
average color value data and exposure calibration data; 
performing chromatic correction of the image using a 
subject failure suppression boundary following the 
2o ■ exposure correction; 

generating image correction data representative of the 
exposure correction and the chromatic correction; and 
applying the image correction data to the image to pro- 
duce a corrected color image. 
25 34. A system for correcting a digital color image scanned 
from film, the system comprising: 
means for producing average color value data for the 
scanned color image; 
30 means for performing exposure correction of the image 
using the average color value data and exposure cali- 
bration data; 

means for performing chromatic correction of the image 
using a subject failure suppression boundary following 
35 the exposure correction; 

means for generating image correction data representative 
of the exposure correction and the chromatic correc- 
tion; and 

40 means for applying the image correction data to the image 
to produce a corrected color image. 

+ * # + + 
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[57] ABSTRACT 

A method and apparatus for processing a character for 
anti-aliased display on a raster output device. A set of 
density values is computed for a set of raster positions to 
represent the character and the density values of the set are 
scaled to extend their range upward toward a maximum 
density value, whereby generally at least one of the density 
values of the set becomes the maximum density value. In 
one embodiment, the set of density values is computed from 
a rendering of the character at resolution higher than the 
resolution of the output device. In another embodiment, the 
character is created by a font having font metrics including 
a reference dimension, and the density values are scaled by 
computing adjusted values as a non-decreasing function of 
the original values, the function being defined to compute a 
maximum adjusted density value for at least one non- 
maximum density value. 
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ADJUSTING CONTRAST EN ANTI-ALIASING 

BACKGROUND OF THE INVENTION 

CROSS REFERENCE TO RELATED 
APPLICATIONS 

The present invention is related to commonly assigned 
U.S. patent application Ser. No. 08/547,562, filed on Oct. 23, 
1995 and entitled METHOD AND APPARATUS FOR 
RENDERING CHARACTERS, the disclosure of which is 
incorporated herein by this reference. 

FIELD OF THE INVENTION 

The invention relates to the computer-implemented ren- 
dering of font characters for display on raster output devices, 
and more particularly, to the display of anti-aliased character 
strokes for small characters. 

BACKGROUND 

An alphabet of characters with a particular design is 
called a "typeface". A "font" is a collection of instructions 
that a processor, such as a microprocessor controlling a 
printer, can use to create text (the characters of the alphabet) 
in a particular typeface. A font used in a computer is 
generally stored in one or more disk files. A font used by a 
printer is generally stored in a read-only memory, down- 
loaded from some computer into printer random access 
memory, or loaded from a disk attached to the printer 
directly or remotely. 

As used herein, a character may be any form of mono- 
chromatic character, number, symbol, icon, graphic, or the 
like that can be displayed as a graphical element on an 
output device. A text character consists of at least one stroke, 
which may be straight or curved and has a nonzero width. 
The specific appearance of a character on an output device 
is created by its font. Present-day fonts, such as PostScript™ 
fonts available from Adobe Systems Incorporated of Moun- 
tain View, Calif., generally include a graphics description 
providing the outline of the character as it is to be displayed, 
an are for that reason referred to as outline fonts. 
a A^^^f/' A character is generally presented on an output device as 
an image consisting of pixels (picture elements) arranged in 
the rows and columns of a raster. If a pixel on the output 
device has only two possible tone values (e.g., a background 
color such as white and a foreground color such as black, for 
normal text documents), the image pixels can be encoded 
each as a single bit. If the output device can have more than 
two tone values at each pixel position (e.g., tones ranging 
from the background color, through blends of the back- 
ground and foreground colors, to the foreground color), 
more than one bit must be used to represent the possible 
pixel values. 

Computers and computer printers generall y scan-convert 
outline font characters for display on raster ou tput devices. 
The result of this interpretation or rendering of the charac- 
tersfesults in a bit map of one-bit pixels or in a run array of 
scan lines indicating the positions of background- 
foreground transitions. The rendering may be thought of as 
providing a set of pixels each representing one of two colors: 
a background color or a foreground color. (The term "pixel" 
is used for both the physical output viewed on a printed page 
or a display monitor, for example, and for the data element 
in a computer; however, the meaning will be clear from the 
context.) For characters elements having edges that are not 
aligned with the raster of the output device, the edges of the 
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displayed image may have a jagged appearance. This effect 
is called aliasing. 

The jagged edges can be smoothed by anti-aliasing tech- 
niques such as those explained in "Computer Graphics, 
Principles and Practice", Second Edition, by James D. Foley 
et al. One type of anti-aliasing that can be performed for 
output devices capable of displaying more than two tones is 
to soften a jagged edge by shading pixels along the edge. 

Anti-aliasing techniques have provided good results for 
large characters. However, they have encountered problems 
with smaller characters. As type size is reduced relative to 
output device resolution, a character's strokes may be 
reduced in width to less than one output pixel. When 
anti-aliasing shades such narrow strokes, the strokes tend to 
15 fade, which can make the resulting output hard to read. 

SUMMARY OF THE INVENTION 

In general, in one aspect, the invention provides a 

20 computer-implemented method for processing a character 
for anti-aliased display on a raster output device. The 
method includes computing a set of density values for a set 
of raster positions to represent the character and scaling the 
density values of the set to extend their range upward toward 

25 a maximum density value, whereby generally at least one of 
the density values of the set becomes the maximum density 
value. In another aspect, the set of density values is com- 
puted from a rendering of the character at resolution higher 
than the resolution of the output device. In another aspect, 

30 the character is created by a font having font metrics 
including a reference dimension, and the density values are 
scaled by computing adjusted values as a non-decreasing 
function of the original values, the function being defined to 
compute a maximum adjusted density value for at least one 

3S non-maximum density value. 

In general, in another aspect, the invention provides a 
computer-implemented method for processing a character 
for anti-aliased display on a raster output device having an 
output pixel position, the character being created at a type 

40 size by a font having font metrics including a reference 
dimension. The method includes rendering the character at 
a resolution higher than the output resolution, computing an 
original density value for the output pixel position from the 
rendering, and co mputi ng an adjusted density value b y 

45 a pplying an adjusfaoent "function to the origi nal density 
value, the adjustmenT^Inction beuig^fined to compute a 
maximum adjusted density value when applied to a range of 
one or more non-maximum original density values. In 
another aspect, the step of rendering the character generates 

50 bit values of a bit map or of a set of run array lines having 
a resolution higher than the output resolution, and the 
original density value is computed as a function of bit values 
generated in the rendering step. In another aspect, the 
original density value is computed as a sum of bit values 

55 generated in the rendering step for positions that correspond 
to the output pixel position. In another aspect, the reference 
dimension is a scalable measure of a standard stem width for 
a vertical or horizontal stem. 
In another aspect, the method includes computing a pixel ^ 

60 value for the output pixel position from the adjusted density ' 
value, where t he pixel value is computedJ ?.v^blending_a 
f oreground color and a background color ac cording to the 
adjusted density value, and where foreground and back- 
ground color are a chromatic color or a gray scale value. In 

65 another aspect, the method includes comparing the scaled 
reference dimension to a threshold value and bypassing the 
step of adjusting the density value if the threshold value is 
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exceeded. In another aspect, the method includes caching 
the adjusted density value for the event that anti-aliasing 
needs to be performed again for the character of the font. In 
another aspect, the adjustment function is a function of the 
original density value and an index value, the adjustment 
function being defined to compute a maximum adjusted 
density value when applied to an original density values that 
is greater than or equal to the index value. In another aspect, 
the adjustment function uses the index value to select a 
precomputed table that maps a density derived from the 
higher-resolution rendering to a value, where the map 
defines a non-decreasing function. In another aspect, the 
index value is approximately a maximum density value 
times the reference dimension scaled to the type size divided 
by a threshold stem width. In another aspect, the threshold 
stem width is in the range of approximately 1.0 to 2.3 pixels. 

In general, in another aspect, the invention provides an 
anti-aliasing method for displaying a character on a raster 
output device having an output resolution, including, in 
general, steps of rendering the character, computing a set of 
density values, scaling the density values, computing pixel 
values for output device pixel positions, and displaying the 
pixel values on the output device. 

Id general, in another aspect, the invention provides a 
computer program storage device — such as a read-only, 
semiconductor memory device, such as an EPROM, or a 
magnetic disk, a magneto-optical disk, or a CD-ROM disk — 
tangibly embodying a set of computer-readable computer 
program instructions including, in various aspects, instruc- 
tions for practicing the methods of the invention as are 
described here. 

The invention has a number of advantages. 

For example, the invention provides small characters that 
are readable and have little fading or distortion. Use may be 
made of the invention without affecting the anti-aliasing of 
large characters and without changing the overall inked 
appearance of a page. 

For a fuller understanding of the nature and further 
advantages of the invention, reference should be made to the 
detailed description taken in conjunction with the accom- 
panying drawings. 

BRIEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorporated in 
and constitute a part of the specification, schematically 
illustrate specific embodiments of the invention and, 
together with the general description given above and the 
detailed description of the embodiments given below, serve 
to explain toe principles of the invention. 

FIG. la is flow diagram of a method of rendering char- 
acters using an anti-aliasing technique according to the 
present invention. 

FIGS, lb and lc illustrate a typical prior art method for 
anti-aliasing a character. 

FIG. 2 illustrates a high resolution bit map for two 
character strokes and their neighboring area. 

FIG. 3 illustrates a density map for the bit map of FIG. 2. 

FIGS. 4 and 5 each illustrate an adjusted density map for 
the bit map of FIG. 2 according to the present invention. 

FIG. 6 is a table illustrating a mapping of original density 
values to adjusted density values according to the present 
invention. 

FIG. la illustrates a high resolution bit map. 
FIG. lb illustrates a density map created from the high 
resolution bit map of FIG. la. 
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FIG. lc is an adjustment map extracted from the table of 
FIG. 6, based on an index value of five. 

FIG. Id is an adjusted density map, resulting from apply- 
ing the adjustment map of FIG. lc to the density map of FIG. 
5 lb. 

FIG. 8 is a diagram illustrating apparatus embodying the 
present invention. 

DETAILED DESCRIPTION 

10 Referring to FIG. 1, in a process for preparing a character 
for display on an output device (such as a monitor or printer) 
according to the present invention, a graphics description 10 
for the character is rendered into a high resolution rendering 
20 (such as a high resolution bit map) having a higher 
resolution than is supported by the output device. (Step 11) 
The graphics description 10 is generally an outline font 
program. In other embodiments, the graph ics descri ption 
may take o^er^fbTmsTincluding^the form ot a hign resolu- 
tion~Mnnap, in which case'ths rc nde nrig step (step ll) is not 

20 necessary and the resolution of the rendering 20 may be 
taken as that given by the graphics description. An outline 
font graphics description rendered to any resolution by a 
rendering program available from a variety of sources, 
including Adobe Systems. As used here, resolution refers to 

25 the spacial density of pixels in an internal computer repre- 
sentation or on an output device, which is often expressed in 
terms of dots per inch. 

From the high resolution rendering 20, a density map 30 

30 is created at the output device resolution. (Step 21) Each 
density element of the density map 30 is computed from a 
plurality of elements (for example, bits) in the high resolu- 
tion rendering 20. In one embodiment, a box filter function 
illustrated in FIGS. 2 and 3, the value of each density 

3S element is computed as the sum of the bit values of the high 
resolution bit map bit positions corresponding to the density 
element. 

The value assigned to a density element may be thought 
of as representing a tone, ranging from a background color 

40 to a foreground color. Having the same resolution, each 
density element of the density map corresponds to a pixel 
position on the output device, and the value of a density 
element may be used to determine the tone at the corre- 
sponding output pixel. For example, for a density element 

45 having a value of zero, the tone of the output pixel may be 
set to a background color; for a density element having a 
maximum value, the tone of the output pixel may be set to 
a foreground color; and for a density element having an 
intermediate value, the tone of the output pixel will be a 

5 0 blend of the background and foreground colors. 

An adjusted density map 40 is created from the density 
map 30 by adjusting the density map values to compensate 
for any fading expected to occur. (Step 31) This step may be 
performed on the density map as a whole after it is 

ss developed, or on parts of it as it is being developed. 

Finally, an output pixel map or pixmap map 50 is com- 
puted from the adjusted density map 40 by assigning a 
corresponding output pixel value to each density element. 
(Step 41) This step may be performed on the adjusted 

60 density map as a whole after it is developed, or on parts of 
it as it is being developed, or in parallel with the step of 
creating a density map. Thus, it is not necessary that the high 
resolution rendering 20 be completed before the creation of 
the density map 30 is begun, nor is it necessary that the 

65 creation of the density map 30 be completed before the 
creation of adjusted density map 40 is begun, nor is it 
necessary that the creation of the adjusted density map 40 be 
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completed before the creation of the output pixel map 50 is a maximum value, but none of those of the narrower stroke 

begun. Moreover, data structures may be used in practicing 44. Thus, if stroke 44 were of the standard stem width, for 

the invention other than the bit maps, pixel maps, and tables example, the adjustment illustrated in FIG. 5 would provide 

that are used here for the sake of illustration. better contrast than the adjustment of FIG. 4. Furthermore, 

Referring to FIGS, lb and lc, in a prior art method for 5 if it were desired to have a pixel of a maximum density value 

anti-aliasing, the character is rendered as a bit map 12 at a Q n each scan line, the adjustment of FIG. 5 would be 

resolution higher than that supported by the output device. selected over that of FIG. 4 for a standard stem width of 

Then, a density map 13 is computed at the same resolution stroke 23 (stroke 53 in FIG 5) 

as the output device (For clarity, zero values are not shown ^ ^ ^ ^ m tioaM te cachedj to 

in the density map.) Each element of the density map is 10 ^ rf ^ J me adjusted density 

computed as a function of numerous bits in the high reso- valu£s for ^ devic< . resoluti even tn6 ou ^ ut 

lution bit map 12, in a process called super sampling. The device ^ d h ch 

specific function used to combine the samples is called the ... ... ..... , 

filter. Then, a device pixel map is computed for output. Each h f ^ ment >° ned > ** density values are 

device pixel corresponds to one density element. The color 1S to incrcase ™"f™» «> situations where anti-aliasing 

or value of the device pixel is computed by blending a « l^ely to result m fading. In one embodiment^, the density 

foreground color and background color according to the tna P 30 15 ad J ustc , d ^ ,l *P * 

, - 4 1 „ ♦ a ** #u n * problem may exist. This situation may be identified by 

corresponding density element. For zero density, the result is K . J . . - , ' * * ./ L 

i . *u T i j i *™ ' a % *u comparing a reference dimension or the character font (such 

equal to the background color. For maximum density, the . , ... , * . . . i_- V 

result is equal to the foreground color, and at intermediate 20 ^ a standard stem width scaled to the type size at which the 

densities the color is a blend of the two. It will be understood 15 bein S re ° dered > to a stroke wdth ^ e 

that the densities described here run linearly from a mini- hold mav * obtamed ln a vanet ? of ^ " may 56 

mum to a maximum value, and that in generating output for predetermined for example, or it may be set by user input. 

a typical color output device, such a color monitor, system * u ^ „ , '° T 15 P J > ™t 

non-linearities must be taken into account to achieve the 2S ™* n threshold value is equal to or greater than the 

desired visual effect. reference dimension scaled to the output type size, the 

_ .... , . .t. values in the original density map are adjusted to increase 

Exemplary bit maps and density maps illustrating the ,, , , B , r . . i 

• 1 . r . . „J„ - , _ J?_ _ the density values assigned to some of the output pixels, as 

present invention are shown in FIGS. 2 through 5. FIG. 2 ^ jg^^^j 

shows a high resolution bit map 22 for two character strokes . , . „ ', . ,. , „ . , , 

23 and 24 and a neighboring area. The circles in the figure 30 * font typically has font metrics, which generally mchide 

represent the bits in a high resolution bit map for the s f alable dimensions such as a standard stem w,dth for a 

character strokes and the crosses define the lower output character which is scaled to the type size at which the font 

resolution supported by an output device. In this example, 15 rendered (Astern m a font is a stroke or a part of a stroke, 

the high resolution bit map has four times the resolution of ^ ned ve [ tlcaU y or horizontally.) A scaled stem 

the output device in both the x and y directions, and each 3S ^ T ^ * scaleds,andard » onz °ntal or vertical stem 

density element and each output pixel of the output device wdth > f a font P^vides a good es unate of stroke 

corresponds to sixteen bits arranged in 4x4 formation on the In u one embodiment, the density values for output 

high resolution bit map. The left stroke 23 has a stroke width P 11 ^ at th f «*? of a character strokc of ! hc standard ™* th 

of 0.5 pixels; the right stroke 24, of 0.25 pixels (at the output are generally adjusted to a maximum density value resulting 

device resolution 1 ) in output characters that retain high contrast, avoid fading at 

_, . . ■ , c 40 small type sizes relative to the output raster resolution, and 

The density maps 33 and 34 shown in FIG. 3 result from ,, ... .. .. u u -.u . .u 

. . . , , are therefore easier to read than they would be without the 

applying a niter function to the graphical elements repre- ,. 

sented in FIG. 2. A variety of filter functions may be used, ,. , , . 

and the specific function used in this example is a box filter D u ensitv ™ P adjustment may be made using a table 60 

function, which gives equal weight to each of the sixteen 45 T ™ 1 f a a wm ^ ma ? S a ° on ^f 

high resolution bit map bits corresponding to a density valu t e an m f™ to ™ ^ icd 

element, and computes a value for that density element as the generaUy diagonal structure, from lower left to upper 

the total number of high resolution bit map bits within the n 8 ht ' of labIe " that m this example the index 

corresponding area. It should be noted, however, that a filter value » the ^ me 35 the l owest on &™ { densU y value that * 

function may take as input overlapping ranges of bits in the 50 mapped l ° 16 > rc P reseDtlD 8 a maximum out P ut 

high resolution bit map, in which case a value of one high Id ^ embodiment, the index is calculated as follows: 
resolution bit can affect more than one density element and 
its corresponding output pixel. 

Referring to FIG. 3, the density values of the adjusted 

density maps 43 and 44 are adjusted from the corresponding 55 
density maps 33 and 34 (FIG. 3) in order to ensure adequate 

contrast in situations where fading is likely to occur. (Step with the result clipped to the range of 2 to 16. (The "scaled 

31) Adjustment can be made by a variety of methods. The stroke width" and "threshold" are both in units of output 

adjustment function used in calculating the values of the device pixels. The scaled stroke width is the standard stroke 

adjusted density map 43 and 44 simply multiplies each 60 width reference dimension (in the units of the character 

original density value by a factor of two, with the result space) scaled to the output type size.) Other functions may 

being limited to a maximum value of sixteen. Similarly, the be used to map the original density values into a larger range 

adjustment function used to create the adjusted density maps of adjusted density values. The particular mapping function 

53 and 54 shown in FIG. 5 multiplies each original density selected will vary depending on the desired result, 

value by a factor of four, with the same limiting maximum 65 This embodiment will be further described with reference 

value of sixteen. Hie adjustment used in FIG. 4 is sufficient to the high resolution rendering illustrated in FIG. la. FIG. 

to cause some density values in the wider stroke 43 to reach 7b illustrates the density map resulting from application of 



index = round 



scaled stroke width + 1/4 

20 x 

threshold + 1/4 
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the box filter function described above. Assuming a scaled 
standard stem width of 0.5 and a threshold of 1.5, the index 
calculated in accordance the function above is 5. For an 
index of 5, the table shown in FIG. 6 defines an adjustment 
function mapping original density values to adjusted density 
values, and the columns of interest have been isolated in the 
adjustment map shown in FIG. 7c. 

FIG. Id shows the result of applying this adjustment to the 
original density map shown in FIG. lb. The adjusted density 
values shown in FIG. Id will result in an output image 
having increased contrast between the tones of character 
strokes of standard width and the background color, thereby 
reducing any fading that otherwise may have occurred. 

The threshold value of 1.5 pixels has been found to give 
pleasing results for a range of fonts for the English alphabet. 
The use of this threshold value is illustrated in FIG. Id. A 
higher value, empirically on the order of 2.25 pixels, gen- 
erally results in standard width strokes having a maximum 
density pixel on each scan line through which the stroke 
passes, regardless. A lower value of approximately 1.0 gives 
that result only for standard width strokes that are substan- 
tially horizontal or vertical. 

Referring to FIG. 8, it will be well understood that the 
methods described here may be readily implemented in 
hardware or in a computer program product tangibly embod- 
ied in a computer program storage device for execution by 
a computer processor. A present-day printer 80 implement- 
ing an interpreter for a page description language, such as 
PostScript, includes a microprocessor 82 for executing pro- 
gram instructions (including font instructions) stored on a 
printer random access memory 84 and a printer read-only 
memory (ROM) 84 and controlling a printer print engine 88. 
The essential elements of a computer are a processor for 
executing instructions and a memory, and these will be 
found in desktop computer 90 and other computers suitable 
for executing computer programs implementing the methods 
described here, which may be used in conjunction with any 
print engine, display monitor, or other raster output device 
capable of producing color or gray scale pixels. Generally, 
a computer will include both a read-only memory and a 
random access memory. Storage devices suitable for tangi- 
bly embodying computer program instructions implement- 
ing the methods described here include all forms of non- 
volatile memory, including semiconductor memory devices, 
such as EPROM, EEPROM, and flash memory devices, 
magnetic disks such as internal hard disks and removable 
disks 92, magneto-optical disks, and CD-ROM disks. 

The present invention has been described in terms of 
specific embodiments. The invention, however, is not lim- 
ited to these specific embodiments. Rather, the scope of the 
invention is defined by the following claims, and other 
embodiments are within the scope of the claims. For 
example, various anti-aliasing techniques may be used with- 
out diminishing the advantages of the present invention. In 
one variation, rather than using a rendering in a higher 
resolution bit map, a run array made up of scan lines at the 
higher resolution that identify the locations of the transitions 
may be used instead. 

What is claimed is: 

1. A computer-implemented method for processing a 
character for anti-aliased display on a raster output device 
where the character is created at a type size by a font having 
font metrics including a standard stem width, the method 
comprising: 



10 



30 



40 
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computing a set of density values to provide one density 
value for each of a set of raster positions to represent 
the character on the raster output device; and 

comparing (i) the standard stem width scaled to the type 
size to (ii) a threshold value, and if the threshold value 
is exceeded increasing at least one of the density values 
in the computed set of density values, thereby com- 
pensating for fading that may occur in the display of the 
character on the raster output device. 

2. The method of claim 1 where the output device has a 
resolution and the set of density values is computed from a 
rendering of the character at resolution higher than the 
resolution of the output device. 

3. The method of claim 1 where 

the density values are scaled by computing adjusted 
values as a non-decreasing function of the original 
values, the function being defined to compute a maxi- 
mum adjusted density value for at least one non- 
maximum density value. 

4. A computer-implemented method for processing a 
character for anti-aliased display on a raster output device 
having an output resolution and one or more output pixel 
positions for display of the character, the character being 
created at a type size by a font having font metrics including 
a reference dimension, the method comprising: 

rendering the character at a resolution higher than the 
output resolution; 

computing a set of original density values, one for each 
output pixel position used to display the character, from 
the rendering, wherein the set of original density values 
includes a highest density value; 

computing an adjusted density value for at least one 
computed original density value by applying an adjust- 
ment function to the original density value, the adjust- 
ment function being defined to compute an adjusted 
density value having a maximum value when the 
adjustment function is applied to the highest density 
value; and 

comparing (i) the reference dimension scaled to the type 
size to (ii) a threshold value, and bypassing the step 
computing an adjusted density value if the threshold 
value is exceeded. 

5. The method of claim 4 where 

the step of rendering the character generates bit values of 
a bit map or of a set of run array lines having a 
resolution higher than the output resolution; and 

the original density value is computed as a function of bit 
values generated in the rendering step. 

6. The method of claim 5 where 

the original density value is computed as a sum of bit 
values generated in the rendering step for positions that 
correspond to the output pixel position. 

7. The method of claim 4 where the reference dimension 
is a scalable measure of a standard stem width for a vertical 
or horizontal stem. 

8. The method of claim 4 further comprising: 
computing a pixel value for the output pixel position from 

the adjusted density value, where the pixel value is 
computed by blending a foreground color and a back- 
ground color according to the adjusted density value, 
and where foreground and background color are a 
chromatic color or a gray scale value; and 
displaying pixel values on the raster output device. 

9. The method of claim 4 further comprising the step of 
caching the adjusted density value for use if anti-aliasing 

needs to be performed again for the character of the 
font. 
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10. A computer-implemented method for processing a 
character for anti-aliased display on a raster output device 
having an output resolution and one or more output pixel 
positions for display of the character, the character being 
created at a type size by a font having font metrics including 5 
a reference dimension, the method comprising: 

rendering the character at a resolution higher than the 
output resolution; 

computing a set of original density values, one for each 
output pixel position used to display the character, from 10 
the rendering, wherein the set of original density values 
includes a highest density value; and 

computing an adjusted density value for at least one 
computed original density value by applying an adjust- 
ment function to the original density value, where the 15 
adjustment function is a function of the original density 
value and an index value and where the index value is 
approximately a maximum density value times the 
reference dimension scaled to the type size divided by 
a threshold stem width, the adjustment function being 20 
defined to compute a maximum value when applied to 
an original density value that is greater than or equal to 
the index value. 

11. The method of claim 10 where 

25 

the adjustment function uses the index value to select a 
precomputed table that maps a density derived from the 
higher-resolution rendering to a value, where the table 
defines a non-decreasing function. 

12. The method of claim 10 where the threshold stem 
width is in the range of approximately 1.0 to 2.3 pixels. 

13. A storage device readable by a machine, tangibly 
embodying a set of computer-readable computer program 
instructions comprising instructions for processing a char- 
acter for anti-aliased display on a raster output device where 35 
the character is created at a type size by a font having font 
metrics including standard stem width, the set of instructions 
comprising: 

instructions for computing a set of density values for a set 
of raster positions to represent the character; and 

instructions for comparing (i) the standard stem width 
scaled to the type size to (ii) a threshold value, and if 
the threshold value is exceeded increasing at least one 
of the density values of the set thereby compensating 45 
for fading that may occur in the display of the character 
on the raster output device. 

14. The apparatus of claim 13 where the output device has 
a resolution and the instructions for scaling the set of density 
values computes from a rendering of the character at reso- 50 
hition higher than the resolution of the output device. 

15. The apparatus of claim 13 where 

the instructions for scaling the set of density values 
compute adjusted values as a non-decreasing function 
of the original values, the function being defined to 55 
compute a maximum adjusted density value for at least 
one non-maximum density value. 

16. A storage device readable by a machine, tangibly 
embodying a set of computer- readable computer program 
instructions comprising instructions for processing a char- 60 
acter for anti-aliased display on a raster output device having 

an output pixel position, the character being created at a type 
size by a font having font metrics including a reference 
dimension, the set of instructions comprising: 6S 
instructions for rendering the character at a resolution 
higher than the output resolution; 



40 



instructions for computing a set of original density values, 
one for each output pixel position, from the rendering, 
wherein the set of original density values includes a 
highest density value; 

instructions for computing an adjusted density value for at 
least one computed original density value by applying 
an adjustment function to the original density value, the 
adjustment function being defined to compute an 
adjusted density value having a maximum density 
value when the adjustment function is applied to the 
highest density value; and 

instructions for comparing (i) the reference dimension 
scaled to the type size to (ii) a threshold value, and 
bypassing the step of computing an adjusted density 
value if the threshold value is exceeded. 

17. The apparatus of claim 16 where 

the instructions for rendering the character generates bit 
values of a bit map or of a set of run array lines having 
a resolution higher than the output resolution; and 

the instructions for computing an original density value 
compute a function of bit values generated by the 
rendering instructions. 

18. The apparatus of claim 17 where 

the original density value is computed as a sum of bit 
values generated in the rendering step for positions that 
correspond to the output pixel position. 

19. The apparatus of claim 16 where the reference dimen- 
sion is a scalable measure of a standard stem width for a 
vertical or horizontal stem. 

20. The apparatus of claim 16 further comprising: 
instructions for computing a pixel value for the output 

pixel position from the adjusted density value, where 
the pixel value is computed by blending a foreground 
color and a background color according to the adjusted 
density value, and where foreground and background 
color are a chromatic color or a gray scale value; and 
instructions for displaying pixel values on the raster 
output device. 

21. The apparatus of claim 16 further comprising: 
instructions for caching the adjusted density value for use 

if anti-aliasing needs to be performed again for the 
character of the font. 

22. A storage device readable by a machine, tangibly 
embodying a set of computer-readable computer Program 
instructions comprising instructions for processing a char- 
acter for anti-aliased display on a raster output device having 
an output pixel position, the character being created at a type 
size by a font having font metrics including a reference 
dimension, the set of instructions comprising: 

instructions for rendering the character at a resolution 
higher than the output resolution; 

instructions for computing a set of original density values, 
one for each output pixel position, from the rendering, 
wherein the set of original density values includes a 
highest density value; and 

instructions for computing an adjusted density value for at 
least one computed original density value by applying 
an adjustment function to the original density value, the 
adjustment function is a function of the original density 
value and an index value and where the index value is 
approximately a maximum density value times the 
reference dimension scaled to the type size divided by 
a threshold stem width, the adjustment function being 
defined to compute a maximum density value when 
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applied to an original density value that is greater than 
or equal to the index value. 
23. The apparatus of claim 22 where 

the adjustment function uses the index value to select a 
precomputed table that maps a density derived from the 



12 



higher-resolution rendering to a value, where the table 
defines a non-decreasing function. 
24. The apparatus of claim 22 where the threshold stem 
width is in the range of approximately 1.0 to 2.3 pixels. 



12/16/2003, EAST Version: 1.4.1 



a. ( „ 

United States Patent m 

Okada 



Illlllllllllillllllllllllll 



US0O5852468A 
[ii] Patent Number: 
[45] Date of Patent: 



5,852,468 
Dec. 22, 1998 



[54] 1-CHIP COLOR VIDEO CAMERA FOR 
GENERATING INTER-PIXEL COLOR 
SIGNAL COMPONENT BY INTERPOLATING 
PRIMARY COLOR SIGNALS FROM 
NEIGHBORING PIXELS 

[75] Inventor: Hidefumi Okada, Daito, Japan 

[73] Assignee: Sanyo Electric Co., Ltd., Japan 

[21] Appl. No.: 604,482 

[22] Filed: Feb. 21, 1996 

[30] Foreign Application Priority Data 

Feb. 27, 1995 [JP] Japan 7-038632 

[51] Int. CI. 6 H04N 3/14; H04N 5/335; 

H04N 9/04; H04N 9/083 

[52] U.S.CI 348/272; 348/280 

[58] Field of Search 348/272, 273, 

348/280, 263, 625, 630, 717, 266, 268, 
269, 277, 281, 282, 320, 321, 322, 323, 
324, 624 

[56] References Cited 

U.S. PATENT DOCUMENTS 
4,591,900 5/1986 Heeb et al 358/44 



4,716,455 12/1987 Ozawa et a! 358/44 

5,280,351 1/1994 Wilkinson 358/140 

5,552,827 9/1996 Maenaka et al 348/266 

5,581,298 12/1996 Sasaki et al 348/222 

FOREIGN PATENT DOCUMENTS 

63-97078 4/1988 Japan . 

Primary Examiner — Andrew I. Faile 

Assistant Examiner — Christopher Onuaku 

Attorney, Agent, or Firm — Michaelson & Wallace; Peter L. 

Michaelson 

[57] ABSTRACT 
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1-CHIP COLOR VIDEO CAMERA FOR 
GENERATING INTER-PIXEL COLOR 
SIGNAL COMPONENT BY INTERPOLATING 
PRIMARY COLOR SIGNALS FROM 
NEIGHBORING PIXELS 

BACKGROUND OF THE INVENTION 

1. Field of the Invention 

The present invention relates to a color video camera. 
More specifically, the present invention relates to a 1-chip 
color video camera having a color separation circuit for 
processing signals from a solid-state image sensor in which 
primary color filters, that is, red, green and blue (hereinafter 
simply referred to as R, G and B) are arranged mosaic-wise. 

2. Description of the Background Art 

As disclosed in the Description of the Prior Art of 
Japanese Patent Laying-Open No. 63-97078 (H04N9/N7), a 
color video camera using 1-chip for primary colors includes 
a solid-state image sensor having a photoreceptor portion, a 
charge transfer portion and a transfer control portion. 

On the photoreceptor portion, a mosaic microfilter is 
deposited. 

More specifically, referring to FIG. 1, the photoreceptor 
portion 85 of solid-state image sensor 1 includes a plurality 
of photoelectric converting elements arranged in a matrix, 
and mosaic elements of the microfilter are correspondingly 
arranged for respective photoelectric converting elements. 
Namely, a microfilter corresponding to one color, for 
example, one of R, G and B is allotted to one photoelectric 
converting element. There are various combinations of filter 
arrangement of R, G and B in such a mosaic microfilter. One 
typical example has a combination in which odd numbered 
rows viewed from below includes GRGR . . . , and even 
numbered rows include GBGB . . . , as shown in FIG. 2. 
More specifically, filters corresponding to green, which 
requires high resolution, are arranged in black squares of a 
checker board, and R and B filters are arranged on white 
squares (hereinafter, referred to as checkerwise 
arrangement). In this case, rows including R filters and rows 
including B filters are alternately arranged. This arrange- 
ment is generally referred to as a Bayer type arrangement. 

From respective pixels of the solid-state image sensor on 
which the microfilter is arranged as described above, color 
signals corresponding to the colors of associated color filters 
are output. The color signals are separated into R, G and B 
color signals respectively, by a color separation circuit in a 
succeeding stage. 

FIG. 1 is a schematic block diagram showing a structure 
of the solid-state image sensor, that is, a charge coupled 
device (hereinafter referred to as CCD) 1. 

CCD-1- includes a photoreceptor portion 85 having a 
plurality of photoelectric converting elements, for example, 
photodiodes, arranged in a matrix corresponding to respec- 
tive pixels; a plurality of vertical transfer registers 83 
receiving charges stored in the diodes corresponding to the 
incident light for transferring charges successively in verti- 
cal direction; a vertical driving circuit 81 for outputting a 
clock pulse voltage for controlling the operation of vertical 
transfer registers 83; a horizontal transfer register 84 for 
receiving charges transferred by respective vertical charge 
transfer registers for horizontally transferring and outputting 
signals by converting the successively transferred signal 
charges to a voltage; and a horizontal driving circuit 82 for 
generating a clock voltage for controlling the operation of 
the horizontal transfer register 84. 



>2,468 

2 

In other words, solid-state image sensor 1 has a structure 
of a so-called interline transfer CCD. 

Therefore, in each pixel at photoreceptor portion 85, light 
intensity of incident light received through the correspond- 

5 ing microfilter is converted to an electrical signal, and output 
as a corresponding analog signal, row by row of pixels. 

When the mosaic microfilter such as described above is 
used, only an R signal is obtained from the pixel on which 
an R filter is disposed, and G and B signals cannot be 

10 obtained. Therefore, the G and B signals of this pixel must 
be generated by interpolation from G and B signals of 
neighboring pixels. 

In a so-called digital camera in which signals from a 
solid-state image sensor are digitized for further processing, 
interpolation of a missing signal has been performed by the 
following operation. 

More specifically, in accordance with the arrangement of 
color filters, a two-dimensional missing signal interpolating 

2Q digital filter performs weighting, using a weight coefficient 
predetermined for each pixel. More specifically, signals, of 
the same color as the missing signal, obtained from adjacent 
neighboring signals are multiplied by respective weight 
coefficients, and resulting multiplied values are added and 

2S then divided by a sum of all weight coefficients, that is, a 
so-called weighted means is calculated, so as to obtain a 
color signal which is of the same color as the missing signal. 

FIG. 2 shows a pattern of arrangement of R, G and B of 
the mosaic microfilter of CCD 1-shown in FIG. 1. In the 

3 q pattern shown in FIG. 2, there are four possible 
arrangements, that is, HI, H2, H3 and H4 shown in FIGS. 
3A, 4A, 5A and 6A, respectively of color filters for a block 
of three pixels by three pixels with an arbitrary pixel 
positioned at the center. 

35 FIG.3Ashowsone(hereinafterreferred to as arrangement 
HI) of four arrangements, in which a G filter is deposited on 
the central pixel. In this case, the G signal obtained from this 
pixel is multiplied by the weight coefficient of "4" as shown 
in FIG. 3B, and then it is divided by "4", so that the G signal 

40 is used as it is, as the G signal of the central pixel. As for the 
R signal, R signals obtained from upper and lower adjacent 
pixels on which R filters are deposited are multiplied by the 
weight coefficient of "2" respectively, as shown in FIG. 3C, 
and the value obtained by adding the R signals of the upper 

45 and lower pixels is divided by "4", whereby the R signal of 
the central pixel is generated. Further, as for the B signal, B 
signals obtained from left and right adjacent pixels on which 
B filters are arranged are multiplied by the weight coefficient 
of "2", respectively, as shown in FIG. 3D, and the value 

50 obtained by adding the B signals of the left and right pixels 
is divided by "4", whereby the B signal for the central pixel 
is generated. 

FIG. 4(A) shows another (referred to as arrangement H2) 
of the four arrangements, in which a B filter is deposited on 

55 the central pixel. Therefore, as for the G signal, G signals 
obtained from upper, lower, left and right four pixels are 
multiplied by the weight coefficient of "1" as shown in FIG. 
4B, and the value obtained by adding the G signals from 
these four pixels is divided by "4". Thus the G signal for the 

60 central pixel is generated. As for the R signal, R signals 
obtained from upper left, upper right, lower left and lower 
right four pixels are multiplied by the weight coefficient of 
"1", respectively, as shown in FIG. 4C, and the value 
obtained by adding the R signals from these four pixels is 

65 divided by "4", whereby the R signal for the central pixel is 
generated. As for the B signal, since a B filter is disposed on 
the central pixel, the B signal obtained from this pixel is 
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multiplied by the weight coefficient of "4" as shown in FIG. 
4D, and the resulting value is divided by a 4", so that the B 
signal is used, as it is, as the B signal of the central pixel. 

FIG. 5A shows a still another one (referred to as arrange- 
ment H3) of the four arrangements, in which an R filter is 
disposed on the central pixel. Therefore, as for the G signal, 
G signals obtained from upper, lower, left and right four 
pixels are multiplied by the weight coefficient of "1" as 
shown in FIG. 5B, and the value obtained by adding the G 
signals from these four pixels is divided by "4", whereby G 
signal of the central pixel is generated. As for the R signal, 
since an R filter is disposed on the central pixel, the R signal 
obtained from this pixel is multiplied by the weight coeffi- 
cient of "4" as shown in FIG. 5C, and the resulting value is 
divided by "4**. Thus the R signal is used, as it is, as the R 
signal for the central pixel. As for the B signal, B signals 
obtained from upper left, upper right, lower left and lower 
right four pixels are multiplied by the weight coefficient of 
"1", respectively, as shown in FIG. 5D, and the value 
obtained by adding the B signals from these four pixels is 
divided by "4", whereby the B signal for the central pixel is 
generated. 

FIG. 6 A shows still another one (referred to as arrange- 
ment H4) of the four arrangements, in which a G filter is 
disposed on the central pixel. As shown in FIG. 6B, the G 
signal obtained from the central pixel is multiplied by the 
weight coefficient of "4", and the resulting value is divided 
by "4", so that the G signal, as it is, is used as the G signal 
for the central pixel. As for the R signal, R signals obtained 
from left and right adjacent pixels on which R filters are 
disposed are multiplied by the weight coefficient of "2**, 
respectively, as shown in FIG. 6C, and the value obtained by 
adding these R signals of the left and right pixels is divided 
by 4, whereby the R signal for the central pixel is generated. 
As for the B signal, B signals obtained from upper and lower 
adjacent pixels on which B filters arc disposed are multiplied 
by the weight coefficient of "2" , respectively, as shown in 
FIG. 6D, and the value obtained by adding the B signals of 
the upper and lower pixels is divided by "4", whereby the B 
signal for the central pixel is generated. 

Such interpolation of color signals as described above is 
performed by means of an interpolation digital filter con- 
sisting of a two-dimensional FIR filter (Finite Impulse 
Response) filter. 

The transfer function H (Z) of the FIR filter with respect 
to the aforementioned weight coefficients are as follows. 
[ARRANGEMENT HI] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G 
SIGNAL) 

H (z)=2hmz- m =l 

(HORIZONTAL DIRECTION OF R SIGNAL) 
H(z)-1 

(VERTICAL DIRECTION OF R SIGNAL) 
H (z)-2hmz- ra -lxz-°+0x2- 1 +lxz- 2 -l+z- 2 
(HORIZONTAL DIRECTION OF B SIGNAL) 
H (z)=l+z" 2 

(VERTICAL DIRECTION OF B SIGNAL) 
Z(zH 
[ARRANGEMENT 112] 
(HORIZONTAL & VERTICAL DIRECTIONS OF G 
SIGNAL) 

H (z)«2hmz- m =lxz-°+2xz- 1 +lxz- 2 =l+2z- 1 +z- 2 
(HORIZONTAL & VERTICAL DIRECTIONS OF R 

SIGNAL) H (z^Xhmz-^olxz-^+Oxz'+lxz-^l+z- 2 
(HORIZONTAL & VERTICAL DIRECTIONS OF B 

SIGNAL) H (z)=l 
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[ARRANGEMENT H3] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G 
SIGNAL) 

H (z)-2hmz- m -lxz-°+2xz- 1 lxz- 2 -l+2z _1 +z- 2 
(HORIZONTAL & VERTICAL DIRECTIONS OF R 

SIGNAL) H (z)=l 
(HORIZONTAL & VERTICAL DIRECTIONS OF B 

SIGNAL) H (z)-l+z" 2 
[ARRANGEMENT H4] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G 

SIGNAL) H (z)=l 
(HORIZONTAL DIRECTION OF R SIGNAL) 
H (z)=l+z~ 2 

(VERTICAL DIRECTION OF R SIGNAL) 
H (z)=l 

(HORIZONTAL DIRECTION OF B SIGNAL) 
H (z)=l 

(VERTICAL DIRECTION OF B SIGNAL) 
H (z)-l+z" 2 

FIG. 7 is a graph showing characteristics of the interpo- 
lation filter as represented by the transfer functions above, in 
which the ordinate indicates gain of the interpolation filter, 
while the abscissa represents operational frequency of the 
interpolation filter. More specifically, for the image pick up 
signal sampled at a prescribed sampling time, the frequency 
plotted on the abscissa corresponds to a reciprocal of the 
period of spatial change in the picked-up image. 

The characteristics of the interpolation filter as repre- 
sented by the transfer functions above for each of the R, G 
and B color signals correspond to the curve PI, P2 and P3 
shown in FIG. 7 with respect to the horizontal and vertical 
directions. 

As for the arrangement Hlshown in FIG. 3 A, the G signal 
can be obtained from the pixel which is at the center both in 
the horizontal and vertical directions, and hence there is not 
the necessity of interpolation. Therefore, the characteristic is 
as represented by the curve PI, which is not dependent on 
frequency. 

As for the R signal, the characteristic is as represented by 
the curve PI in the horizontal direction, since interpolation 
from left and right pixels is not necessary. However, in the 
vertical direction, it is interpolated by using R signals of 
upper and lower adjacent pixels. Therefore, the character- 
istic is as shown by the curve P3 which lowers toward one 
half (hereinafter referred to as Vl Nyquist frequency) of 
Nyquist frequency Nq, which is the sampling frequency, and 
has an alias component in the frequency range higher than 
Vi Nyquist frequency. 

Further, the B signal has the characteristic as represented 
by the curve P3 in the horizontal direction as it is interpo- 
lated by B signals from left and right pixels, and in vertical 
direction, the characteristic is as represented by the curve 
PI, since it is not interpolated from upper and lower adjacent 
pixels. 

As for the arrangement H2 of FIG. 4A, the G signal in the 
horizontal direction has the characteristic as represented by 
the curve P2 in which gain of high frequency component 
lowers because of the characteristic of a two-dimensional 
FIR filter as the G signals of left and right adjacent pixels as 
well as upper and lower adjacent pixels contribute to inter- 
polation. It has the characteristic as represented by the curve 
P2 also in the vertical direction, since G signals from upper 
and lower adjacent pixels as well as left and right adjacent 
pixels contribute to interpolation. 

As for the R signal, there are no pixels contributing to 
interpolation in the central column, and hence in horizontal 
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direction, it cannot help but depend on the left and right Another object of the present invention is to provide a 

columns. Therefore, the characteristic is as shown by the 1-chip color video camera capable of minimizing difference 

curve P3. Similarly, in vertical direction, there is not a pixel between frequency characteristics of R and B signals and the 

contributing to interpolation in the central row, and hence it frequency characteristic of the G signal, 

cannot help but depend on upper and lower rows. TTierefore, 5 A ^ objec( f ft invention is to provide 

it also has the characteristic as represented by the curve P3. - , ., J . . ; 

. • • * • *• jl a 1-chip color video camera in which primary colors the 

The B signal docs not require interpolation, and hence it - r . . . - r . . J . , 

has the characteristic as represented by the curve PI both in same frec l uenc y characteristics for every pixel of the sohd- 

the horizontal and vertical directions. state una S e sensor - 

The arrangement H3 shown in FIG. 5A will be described. *n summary, the present invention provides a 1-chip color 

As in arrangement H2, the G signal has the characteristic as 10 video camera including a solid-state image sensor and an 

represented by the curve P2 both in the horizontal and interpolating circuit. 

vertical directions. The solid-state image sensor includes photoelectric con- 

The R signal, which needs no interpolation, has the verting elements, corresponding to respective pixels, 

characteristic as represented by the curve PI both in the arranged in an array. The solid-state image sensor includes 

horizontal and vertical directions. « a fiUer array m wbich primary color filters are disposed 

As for the B signal, it has the characteristic as represented m a prescr ibed arrangement, corresponding to the photo- 

by the curve P3 both in the horizontal and vertical directions, electric Qom ^ n]n% elements, on a photoreceptor surface 

as does the R signal of arrangement H2. side ^ interpolating circuit receives an output from the 

The arrangement H4 shown in FIG. 6A will be described. solid-state image sensor and outputs a corresponding color 

The G signal, which needs no interpolation, has the char- 20 sigQal ^ mchldes a paralld signa] QUtput 

actenstic as represented by the curve PI both in the hori- circuit> a coatrol circuit> ^ a color separation circuit. The 

zontal and vertical directions. parallcl co , 0f signal Qutput ckcu[i rcc6ives output from the 

The R signal has the characteristic as represented by the solid-state image sensor and successively outputs a color 

curve P3 in the horizontal direction, and has the character- signal corre sponding to a prescribed even number of rows of 

istic as represented by the curve PI in the vertical direction, 2 * pixels 5y in paral]el and in synchronization, 

as does the B signal in arrangement HI. ^ circuit out p Uts a synchronized interpolation 

Further, the B signal has the characteristic as represented designating signal in accordance with correspondence 

by the curve PI in the horizontal direction, and the charac- between the color signals output column by column from the 

teristic as represented by the curve P3 in the vertical paralJel color signal output circuit ^ Ae arrang e m eDt of 

direction, as does the R signal of arrangement HI. 30 predetermined weight coefficients for the prescribed 

In this manner, in the conventional interpolation filter, R, arra ngement of the color filter array. The color separation 

G and B signals have filter characteristics different from circuit rcceivcs outputs from tnc paralle i color signal output 

each other in accordance with the arrangement of the color ckcuit arjd> m accorc iance with the interpolation designating 

filters. If the gain of the interpolation filter at Vi Nyquist signal> performs interpolation of color signals from a pixel 

frequency much differs from the gain near the Nyquist 35 block including prescribed even numbered rows and pre- 

frequency Nq in FIG. 7, there would be significant color even numbered columns of pixels, and outputs 

moire near such frequencies. successively and, in synchronization, a color signal corre- 

In order to suppress such color moire in the 1-chip color spondi ng to the central position of the pixel block, 

video camera, an optical lowpass filter (LPF) is placed in an r , ... . ^ , - , 

. . . r- .j . ™/™ . , , in In accordance with another aspect, the 1 -chip color video 

optical path of incident light to CCD 1. The optical lowpass w _ • i j rj . . • *, 

-f r t . , r , K camera mcludes a solid-state image sensor and an mterpo- 
filter removes high frequency component before sampling 

. , ^ J L .. t lating circuit, 

by the CCD 1, so as to reduce the alias component at the ~~ 

sampling, and hence color moire can be suppressed. The sohd-state image sensor includes photoelectric con- 
However, that the high frequency component of the incident ver,ln e dements corresponding to respective pixels, 
light that is removed induces, at the same time, lower « arranged in an array. The solvate image sensor mcludes 
resolution a co ^ or ^ ter a^ay in which primary color filters are arranged 

Further,' dependent on the arrangement, the G signal, mosaic-wise corresponding to the photoelectric converting 

which among primaries, contributes most to brightness elements on a photoreceptor surface side. The color filter 

would have such a characteristic as represented by the curve arra y |" cludes ' m , a co,or filter ^ngement of arb.trary two 

P2 which suffers from significant attenuation in the high so rows by two columns, green filters arranged in dtagonal 

frequency range, which results in degraded resolution. direct™. The interpolating circuit receives an output from 

As for of color niters themselves, the color filter of three mc sohd-state image sensor and interpolates a green signal 

primaries such as described above but color niters of component at the central portion of a pixel block corre- 

complementary colors may be used in lieu of the color filers s P° ndm S to °° lor filter arrangement of two 

of three primary colors such as described above. However, « rows by two columns, by mean vahie of green s.gnals 

in view of color reproduction property, generally primary from P«»«o*ctnc converting elements corre- 

color filters are superior to complementary color filters. spading to the green filters arranged m the diagonal direc- 

Therefore, a method in which color signals are interpolated tl0D " 

by using color filters of primary colors, rather than comple- According to a still further aspect of the present invention, 

mentary colors, is desirable. 60 me 1-chip color video camera includes a solid-state image 

sensor and an interpolating circuit. The solid-state image 

SUMMARY OF THE INVENTION sensor includes photoelectric converting elements, corre- 

An object of the present invention is to provide a 1-chip sponding to respective pixels, arranged in an array. The 

color video camera that provides a frequency characteristic solid-state image sensor includes a color filter array in which 

that exhibits relatively little attenuation of gain up to a high 65 primary color filters are arranged mosaic-wise correspond- 

frequency range, for the G signal which contributes, of three ing to the photoelectric converting elements, on a photore- 

primary colors, most to brightness. ceptor surface side. The color filter array includes red and 
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blue filters arranged checkerwise, and rows including red FIGS. 6 A to 6D are illustrations showing allotment of 

filters and rows including blue filters are alternately weight coefficients for interpolation in a conventional color 

arranged. The interpolation circuit interpolates each of red video camera, in which FIG. 6A shows filter arrangement 

and blue color signal components at the central portion of a H4, FIG. 6B shows an arrangement of weight coefficients for 

pixel block, based on the signals from the pixel block 5 the G signal, FIG. 6C shows an arrangement of weight 

including 4 rows by 4 columns of pixels, that is, 16 pixels. coefficients for the R signal and FIG. 6D shows an arrange - 

The. interpolating circuit includes a two-dimensional non- raent of weight coefficients for the B signal, 

recursive digital filter circuit for switching column-wise pjc. 7 is a graph showing frequency characteristics of 

values of vertical sums of weight coefficients between (0, 3, mree primary signals after interpolation in the conventional 

0, 1) and (1, 0, 3, 0), in accordance with the arrangement of 10 co j or video camera. 

color filters corresponding to the sixteen pixels. FIG. 8 is a block diagram showing a structure of a color 

According to a still further aspect of the present invention, signa i p roce ssing circuit 100 in accordance with one 

the 1-chip color video camera includes a solid-state image embodiment of the present invention. 

sensor and an interpolating circuit. FIG. 9 shows a position of interpolation portion in a pixel 

The solid-state image sensor includes a color niter array block m accoldance with one embodiment of the present 

in which primary color filters are arranged mosaic-wise invention 

corresponding to photoelectric converting elements on a j ^ illustrations showing allotment of 

photoreceptor surface side. The interpolating circuit gener- . , 4 «- ■ . r . * r ■ T i • 

, t. i . r i -i * * weight coefficients for arrangement 1 for interpolation in 

ates a plurality of color signal components at a position ° , < & A fll _ * • *■ ■ 

l*a ~iu l ic • i • u • . i j i j- ** 2n accordance with one embodiment of the present invention in 

shifted by half a pixel in horizontal and vertical directions ^ . L ^i. , l 

- .i_ . c _i_ •* • i • . . . which FIG. 10A shows filter arrangement, FIG. 10B shows 

from the center of an arbitrary pixeL based on color signal A - . , t ~. . A & - ' ' . . ™~ 

. c t f . , °^ arrangement of weight coefficients for the G signal. FIG. 

component of a plurality of neighboring pixels. irk ^ , & A . . . . 4 ' _ 

r r / o r jqxj shows arrangement of weight coefficients for the R 

Therefore, an advantage of the present invention is that signal> ^ 10D shows of weight coeffi . 

the G signal, which contributes most to brightness, exhibits cients for the B signal 
a frequency characteristic that suffers relatively little attenu- 25 

«. »v . • . c t. t • * . rlGo. llAto 11D are illustrations showing allotment ot 

ation up to the high frequency range; hence high resolution . r™ 1. „ mu^au^ ^ 

becomes possible weight coefficients for arrangement 2 for interpolation in 

, , , , . ... accordance with one embodiment of the present invention in 

Another advantage of the present invention is that gen- which nG 11A shows flltef FIG UB shows 

eration of a color false signal is suppressed, since difference 30 arrangement of weight coefficients for the G signal, FIG. 

between the frequency characteristics of R and B signals and uc ^g^,,, of weighl coefficien ts for the R 

that of G signal is small at % Nyquist frequency. signal> ^ na UD shows arrangemem of weight coeffi . 

A still another advantage of the present invention is that cients for the B signal 

same frequency characteristics can be provided for every nGS UA tQ ^ m iUustrations showin aIlotment of 

pixel with respect to each of the three primary colors. 35 coefficients for arraDgemeQt 3 for mterpo i ation m 

The foregoing and other objects, features, aspects and accordance with one embodiment of the present invention in 

advantages of the present invention will become more which pjQ i 2A shows filter arrangement, FIG. 12B shows 

apparent from the following detailed description of the arrangemen t of weight coefficients for the G signal, FIG. 

present invention when taken in conjunction with the nQ arrangemeat of weight coe ffi c ients for the R 

accompanying drawings. 40 s j gna i, and FIG. 12D shows arrangement of weight coeffi- 

BRIEF DESCRIPTION OF THE DRAWINGS cients for the B signal. 

FIG. 1 is a schematic block diagram showing a structure FIGS. 13A to 13D are illustrations showing allotment of 

of a conventional CCD 1. weight coefficients for arrangement 4 for interpolation in 

FIG. 2 is a schematic diagram showing a structure of a 4 , accordance with one embodiment of the present invention in 

mosaic filter disposed on a conventional CCD 1. which 13A shows mter arrangement, FIG. 13B shows 

FIGS. 3A to 3D are illustrations showing allotment of *™igpment of weight coefficients for the G signal, FIG. 

weight coefficients for interpolation in a conventional color 13C f 0 ™*™^™* of weight cocffici r ents f ° r the * 

video camera, in which FIG. 3A shows filter arrangement signal and FIG. 13D shows arrangement of weight coeffi- 

Hl, FIG. 3B shows an arrangement of weight coefficients for 50 cients for mc B Sl ^' 

the G signal, FIG. 3C shows an arrangement of weight FIG. 14 shows position of interpolation portion on the 

coefficients for the R signal and FIG. 3D shows an arrange- COT in accordance with one embodiment of the present 

ment of weight coefficients for the B signal. invention. 

FIGS. 4A to 4D are illustrations showing allotment of FIG. 15 shows frequency characteristics of three primary 

weight coefficients for interpolation in a conventional color 55 signals in accordance with one embodiment of the present 

video camera, in which FIG. 4A shows filter arrangement invention. 

H2, FIG. 4B shows an arrangement of weight coefficients for FIG. 16 shows switching control of a selector circuit in a 

the G signal, FIG. 4C shows an arrangement of weight color signal processing circuit 100 in accordance with one 

coefficients for the R signal and FIG. 4D shows an arrange- embodiment of the present invention, 
ment of weight coefficients for the B signal. so 

FIGS. 5 A to 5D are iUustrations showing allotment of 
weight coefficients for interpolation in a conventional color 
video camera, in which FIG. 5 A shows filter arrangement [> FIG. 8 is a block diagram showing a structure of a color 

H3, FIG. 5B shows an arrangement of weight coefficients for signal processing circuit 100 including components from a 

the G signal, FIG. 5C shows an arrangement of weight 65 CCD 1 as an image sensor up to a color separation circuit 8, 

coefficients for the R signal and FIG. 5D shows an arrange- in a color video cameral in accordance with one embodiment 

ment of weight coefficients for the B signal. of the present invention. 
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The incident light is formed on CCD 1 by means of a lens Color separation circuit 8 includes ten 1 clock delay 

(not shown), and photo-electrically converted to an image providers 10, 11, 12, 13, 14, 15, 16, 17, 18, and 19 for 

signal. A microfilter 70 including R, G and B color filters providing a delay of 1 clock for the input signal; four 

arranged mosaic-wise is provided on the photoreceptor multipliers 20, 21, 22, and 23 for multiplying the value of an 

surface of CCD 1. It is assumed that the arrangement of 5 mput s ign a i by 2; fourteen selectors 24, 25, 26, 27, 28, 29, 

respective color filters of mosaic microfilter 70 is the same 30 31 32> ^ 34 35^ 36 37 for selecting one of two 

as the prior art example shown inFlG^ The light which has mput signals; and five adders 38, 39, 40, 41 and 42 for 

passed through the lens is supplied through microfilter 70 to addi ^ • m si . 

photoreceptor portion of CCD-1. In accordance with the ^ , ><* j ^ j r« n . 

intensity of incident light received through the filter, charges 1ft . ^ structure will be more specifically desenbed. The first 

stored in photoreceptor portion 85 (see FIG. 1) for one field 10 *put signal from A/D converter 4 is mput to 1 clock delay 

period are transferred by vertical transfer register 83 and provider 10, and the output from delay provider 10 is 

horizontal transfer register 84 to be output externally from directly input to terminal 246 of selector 24 as well as to 1 

CCD 1. clock delay provider 14. The output from 1 clock delay 

More specifically, CCD 1 includes photoreceptor portion provider 14 is input to a terminal 24a of selector 24. 

85; vertical transfer register 83 for vertically transferring an 15 The second input signal which is an output signal from 1H 

output in accordance with the light intensity received at delay provider 5 is directly input to a terminal 256 of 

photoreceptor portion 85; horizontal transfer register 84 selector 25, as well as to 1 clock delay provider 11. The 

arranged at a terminating end of the vertical transfer register output from 1 clock delay provider 11 is input to 1 clock 

for transferring charges transferred from the vertical transfer de t av provider 15 of the succeeding stage, as well as to 

register in a horizontal direction, a vertical driving circuit 81 20 multiplier 20. The output from 1 clock delay provider 15 is 

for receiving a vertical synchronizing signal, a horizontal m P ut to 1 clock dela y provider 18 of the succeeding stage, 

synchronizing signal and a clock signal of a fixed frequency « we 11 as t0 multiplier 21. The output from 1 clock delay 

for outputting a vertical transfer pulse to cause vertical provider 18 is input to terminal 25a of selector 25. 

transfer register 83 to execute charge transfer; and a hori- The third input signal, which is the output from 1H delay 

zontal driving circuit 82 for receiving similar signals as 25 provider 6 is directly input to terminal 266 of selector 26, as 

vertical driving circuit 81 for outputting a horizontal transfer well as to 1 clock delay provider 12. The output from 1 clock 

pulse for driving charge transfer by horizontal transfer delay provider 12 is input to 1 clock delay provider 16 of the 

register 84. In synchronization with the vertical synchroniz- succeeding stage as well as to multiplier 22. The output from 

ing signal, an output corresponding to the light intensity ^ Q 1 clock delay provider 16 is input to 1 clock delay provider 

received at photoreceptor portion 85 is read to vertical 19 of the succeeding stage, as well as to multiplier 23. The 

transfer register 83, and in the period of the horizontal output from 1 clock delay provider 19 is input to a terminal 

synchronizing signal, charges are transferred vertically row 26a of selector 26. 

by row in vertical transfer register 83. In accordance with the The fourth input signal, which is the output from 1H delay 

clock signal period, charges are transferred column by ^ provider 7 is input to 1 clock delay provider 13. The output 

column in a horizontal direction in horizontal transfer reg- from delay provider 13 is directly input to a terminal 27b of 

ister 84. selector 27 as well as to 1 clock delay provider 17. The 

Such driving operation of the CCD 1 as described above output from 1 clock delay provider 17 is input to a terminal 

is a well known operation for a so-called interline type CCD 27a of selector 27. 

1. The vertical and horizontal synchronizing signals as well 4Q The output from selector 24 is input to terminals 32a and 

as the clock signal are output from a timing pulse generating 34a of selectors 32 and 34, respectively, of the succeeding 

circuit 71 shown in FIG. 8. stage. The output from selector 25 is input to terminals 326 

Again referring to FIG. 8, the image signal output from and 346 of selectors 32 and 34, respectively. The output 

CCD 1 is subjected to known noise removal operation in a from selector 26 is input to terminals 33a and 35a of 

correlated double sampling circuit (hereinafter referred to as 4S selectors 33 and 35, respectively of the succeeding stage. 

CDS circuit) 2, amplified by an auto gain control circuit The output from selector 27 is input to terminals 336 and 

(hereinafter referred to as AGC circuit) 3, and converted to 356 of selectors 33 and 35, respectively, 

a digital signal in an A/D converter 4. The output from multiplier 20 is input to terminals 28a 

The digital image signal is applied as a first input signal and 30a of selectors 28 and 30, respectively, of the succeed- 
directly to a color separation circuit 8, which is a two- 50 kg stage. The output from multiplier 21 is input to terminals 
dimensional non-recursive digital filter, as well as to a scan 286 and 31a of selectors 28 and 31, respectively, of the 
line delay provider (hereinafter referred to as 1H delay succeeding stage. The output from multiplier 22 is input to 
provider) 5. An output from 1H delay provider 5 is input as terminals 29a and 306 of selectors 29 and 30, respectively, 
a second input signal to color separation circuit 8 as well as of the succeeding stage. Hie output from multiplier 23 is 
to a 1H delay provider 6 of a succeeding stage. Further, an 55 input to terminals 296 and 316 of selectors 29 and 31, 
output from 1H delay provider 6 is input as a third input respectively, of the succeeding stage, 
signal to color separation circuit 8 as well as to a 1H delay The outputs from selectors 32 and 33 are added in adder 
provider 7 of a succeeding stage, and an output from 1H 38 of the succeeding stage, and outputs from selectors 34 
delay provider 7 is input, as a fourth input signal, to color and 35 are added in adder 39 of the succeeding stage. The 
separation circuit 8. 60 output from selector 28 is input to terminals 36a and 37a of 

Therefore, the first to fourth four input signals correspond selectors 36 and 37, respectively, of the succeeding stage, 

to image signals of four scanning lines (four lines), and the The output from selector 29 is input to terminals 366 and 

signals from the four lines are collectively input to color 376 of selectors 36 and 37, respectively, of the succeeding 

separation circuit 8. stage. The outputs from selectors 30 and 31 are added in 

Thus, an FIR (Finite Impulse Response) filter is imple- 65 adder 40. 

mented by color separation circuit 8 and three 1H delay The output from adder 38 is input, together with the 

providers 5, 6 and 7. output from selector 36, to adder 41, and added therein. The 
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output from adder 39 is input, together with the output from S33 are output from 1 clock delay providers 18, 15 and 11, 

selector 37, to adder 42, and added therein. respectively, and signal S34 is output from 1H delay pro- 

The output from adder 41 would finally be the R signal vider 5 - Signals S42 and S43 are output from 1 clock delay 

which has gone through color separation processing; the providers 14 and 10, respectively, and signal S44 is output 

output from adder 42 would be the B signal, and the output 5 from A/D converter 4. 

from adder 40 would be the G signal. In the structure of color separation circuit 8 described 
In the above described flow of signal processing, it atove, m order to perform mterpolation in color separation 
becomes possible to process signals of four successive circuit 8 > a P ixel block consisting of 16 pixels, that is, 4x4, 
pixels of a corresponding line, as three 1 clock delay OQ CCD 1 must ^ FIG - 14 snows positions where 
providers are arranged in series for each input signal. Signals 10 interpolation takes place with respect to the arrangement of 
of a 4x4 pixel block can be handled if three 1 block delay thc mosaic color filter shown in FIG. 2. As already 
providers are connected in series for each input signal. described, 4x4, that is, 16 pixels are necessary for interpo- 
Switching of fourteen selectors is controlled in the following lation - Therefore, as shown in FIG. 14, the result of inter- 
manner by a switch control signal from switch control polation when the image signals from pixels of lowermost 
circuit 72 15 & TCC rows m output is meaningless. Further, interpolation 

Assume that a 4x4 pixel block of CCD 1 includes 16 f f "V* °f thre « ^ eis ° f lhe fou " b ™ w 

pixels represented by Pll to P44 as shown in FIG. 9, for * om **b°nom "f^y™*™^ M ° re ^"ficaUy, 

r . r - . J t . 0 * n o j i> the result of interpolation at interpolation portion Y comes to 

example. Color separation circuit 8 generates R, G and B , f r . n „ A . r . J r * , c 

signate at the central portion M denoid by a hatched circle ,„ have , meani ?S ?» r ^ whea ^ ^ s, S m1 ° f 

of the shown in FIG. 9 by interpolation of R, G and B signals 20 P ucl X 15 obtamcd > m FIG - 14 

from sixteen neighboring pixels. Therefore, switching control of respective selectors for 

c n i .• • •.■> . n interpolation must be started from the time point when the 

More specifically cobr separaton circuit 8 generates R, ^ bom ^1 X is output from CCD 1. 

G and B signals at a position obtained by shifting four . 5 , , 

central pixels of the 4x4 pixel block by half a pixel in 25 r He*, i *™ « Possible arrangements as shown in 

horizontal and vertical directions, in other words, the posi- F * GS " 10 ***** 4x4=16 P ixels for the microcolor filters 

tion obtained by offsetting the four central pixels by half a shoWD m FIG * 14 

pixel in horizontal and vertical directions, by using R, G and FIGS. 10A to 13A show arrangements of color filter 

B signals from some of the neighboring sixteen pixels. arrays of 4x4=16 pixels. Figs. 10B to 13B show weight 

Consider sixteen pixels shown in FIG. 9. First, signals 30 efficients for respective pixels when the G signal at the 

from lowermost one line of pixels are successively output cen ^ P ouU 15 to * e S en <l rated b y ^erpolation. FIGS. 10C 

from horizontal transfer register 84 of CCD 1 at every 1 «? " C show wei f ht coefficients for respective pixels when 

clock, namely, Pll - - P12 - - P13 - - P14. When output me "J* c f n n ' ral ^ on u 15 10 ^ generated by 

of the signals of all the pixels in this line is completed, interpolation. FIGS. 10D to 13D show weight coefficients 

signals from pixels of the second lowest line are output, 35 ? or respective P ix els when the B signal at the central position 

namely, P21 - ^ P22 - - P23 - - P24. 15 t0 bc S cncrated bv interpolation. 

^ * . , - . . . , c . . Referring to FIG. 14, in the process of interpolation at 

Thereafter, signals from the third from the bottom line are . . . & . v * *, ~ . , . Ae _ 

output namelv P31 - -* P32 - ^ P33 - -> P34 mterpolation portion Y performed as a G signal is read from 

p ' ' pixel X, 16 pixels surrounding interpolation portion Y at the 

When the output of signals from the pixels of the third line center would be as shown in FIG. 10A. Therefore, for the 

from the bottom is completed, signals from respective pixels interpolation of interpolation portion Y, switching control 

in the uppermost line are successively output, namely, P41 appropriate for arrangement 1 of respective selector circuits 

*P47 _ pa* > P44. , 

rt^ -~ rfo rt*». becomes necessary. 

For convenience of description, in the following, the when B signal is read from a pixel next to the pixel X in 

signal obtained by photoelectric conversion from pixel Pll 45 me interpolation at an interpolation portion adjacent to 

will be denoted by Sll, the signal from pixel P12 will be interpolation portion Y on the right becomes possible. The 

denoted by S12 and similarly, signals output from pixels up arrangement of sixteen pixels surrounding this interpolation 

to P44 will be denoted by the reference characters up to S44. portion corresponds to arrangement 2 and hence switch 

The image signals Sll to S44 are passed through CDS control appropriate for arrangement 2 of respective selector 

circuit 2 and AGC circuit 3 and successively converted to 50 circuits becomes necessary. 

digital values by A/D converter 4. Thereafter, as long as image signals are output from the 

By the time the output of four lines is completed and the pixels of this line, arrangement of sixteen pixels surrounding 

signal S44 from pixel P44 is output from A/D converter 4, each interpolation portion would be arrangements 1 and 2 

one line of signals, that is image signals Sll to S14 are input switched alternately. Therefore, relative to this switching, 

to color separation circuit 8 from 1H delay provider 7, and 55 selectors must be controlled to be switched to the state 

one line of signals, that is, image signals S21 to S24 are input suitable for the respective arrangements, 

to color separation circuit 8 from 1H delay provider 6. As for the next line, that is, a line above the line to which 

Similarly, one line of signals, that is, image signals S31 to pi^l S belongs, at a time point when R signal is read from 

S34 are input to color separation circuit 8 from 1H delay the fourth from the left pixel, interpolation at an interpola- 

provider 5, and one line of signals, that is, image signals S41 6 o tion portion adjacent to interpolation portion Y on the line 

to S44 are directly input to color separation circuit 8. above becomes possible. The arrangement of sixteen pixels 

Therefore, signal S12 is output from 1 clock delay pro- surrounding this interpolation portion corresponds to 

vider 17, signal S13 is output from 1 clock delay provider arrangement 3 shown in FIG. 12A. Therefore, selector 

13, and signal S14 is output from 1H delay provider 7. circuits must be switched to be suitable for arrangement 3. 

Similarly, signals S21, S22 and S23 are output from 1 clock 65 Further, when reading of the G signal from the adjacent right 

delay providers 19, 16 and 12, respectively, and signal S24 pixel is completed, interpolation at interpolation portion 

is output from 1H delay provider 6. Signals S31, S32 and adjacent to interpolation portion Y on upper right position 
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becomes possible. The arrangement of sixteen pixels sur- 
rounding this interpolation portion corresponds to arrange- 
ment 4 shown in FIG. 13A. Therefore, selector circuits must 
be switched to be suitable for arrangement 4. 

Thereafter, while the image signals are output from the 
pixels of this line, the arrangement of sixteen pixels sur- 
rounding the corresponding interpolation portions corre- 
spond to arrangements 3 and 4 switched alternately. 
Therefore, selector circuits must be controlled to be 
switched to suitable states for respective arrangements, 
correspondingly. 

In the next row, after the point in time at which the image 
signal from the fourth pixel is input, switching control 
suitable for arrangements 1 and 2 is executed. Similarly, in 
the next row, after the point in time at which the image signal 
from the fourth pixel is input, switching control suitable for 
arrangements 3 and 4 is executed. 

Thereafter, control for switching arrangements 1 and 2 
alternately and arrangements 3 and 4 alternately for every 
other row is continued until reading of image signals from 
the pixels of the uppermost row is completed. 

When reading of image signals of the one image plane is 
completed, stored charges at the photoreceptor portion are 
again read to the vertical register and reading from the 
lowermost line of pixels is again performed, similar switch- 
ing control as described above is repeated. 

In order to control switching of respective selectors as 
described above, actually, it is necessary to determine the 
position of the pixel on the CCD 1 from which image signal 
is being output, by means of a vertical counter and a 
horizontal counter included in switch control circuit 72 (see 
FIGS. 8 and 10A-13D). 

The vertical counter is reset by the vertical synchronizing 
signal, counts the horizontal synchronizing signal and hence 
counts the line to which the pixel being processed belongs. 
The horizontal counter is reset by the horizontal synchro- 
nizing signal, counts the clock signals synchronized with 
horizontal transfer, and determines to which column in the 
horizontal direction the pixel being processed belongs. By 
these two counters, that is, vertical and horizontal counters, 
the position of the pixel from which an image signal is being 
output on the CCD 1 is determined. For example, if it is 
determined that the image signal from a pixel at the fourth 
row from the bottom and fourth column from the left is 
being output from CCD 1, switch control circuit 72 deter- 
mines that the arrangement of 4x4=16 pixels used for 
interpolation corresponds to arrangement 1, and outputs a 
switch control signal corresponding to arrangement 1. As 
long as the output of pixels belonging to this line continues, 
switch control signals corresponding to arrangements 1 and 
2 are output alternately in the period of clock signal. 

If it is determined by the vertical and horizontal counters 
that the position on the CCD 1 of that pixel from which 
image signal is being output is the pixel at the fifth row from 
the bottom and fourth column from the left, switch control 
circuit 72 determines that the arrangement of 16 pixels used 
for interpolation corresponds to arrangement 3 and outputs 
a switch control signal corresponding to arrangement 3. As 
long as the output of image signals from pixels belonging to 
this line continues, switch control signals corresponding to 
arrangements 3 and 4 are output alternately in the period of 
the clock signal. 

The image signal output from the corresponding pixel on 
the CCD 1 passes through CDS circuit 2, AGC circuit 3 and 
A/D converter 4, before it is input to color separation circuit 
8. Therefore, after the delay of time necessary for these 
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processings, the switch control signal is output from switch 
control circuit 72 to respective selector circuits. The clock 
signal is in synchronization with charge transfer by hori- 
zontal transfer register 84 of CCD 1, and it also serves as a 
5 driving clock of color separation circuit 8. 

FIG. 16 shows correspondence between switch control 
signals from switch control circuit 72 and the arrangement 
of sixteen pixels. 

In accordance with the corresponding pixel arrangement 
10 of the image signals input to color separation circuit 8, 
selector circuits 24 to 37 are switched to select their corre- 
sponding inputs (a or b) as shown in FIG. 16, and interpo- 
lated R, B and G signals are output from color separation 
circuit 8. 

15 The color separation operation in the color separation 
circuit 8 described above is as follows, with reference to 
FIGS. 8-13D. 

First, the procedure for generating G signal will be 
described. When a G signal is to be generated at interpola- 
20 tion portion Y which is the central portion of 4x4=16 pixels 
at which interpolation takes place, G signals obtained from 
two of four pixels surrounding interpolation portion Y are 
used. 

25 More specifically, in the arrangement 1 shown in FIG. 
10A and in the arrangement 4 shown in FIG. 13A, G signals 
from pixels P22 and P33 are multiplied by the weight 
coefficient of "2" as shown in FIGS. 10B and 13B, and the 
multiplied values are added, whereby the G signal at the 

3Q interpolation portion Y is generated. 

In the arrangement 2 shown in FIG. 11A and in the 
arrangement 3 shown in FIG. 12A, G signals from pixels 
P32 and P23 are allotted with weight coefiBcient of "2" as 
shown in FIGS. 11B and 12B, and similar calculation is 

3S performed to generate the G signal for the interpolation 
portion Y. 

In the above described processing, interpolation is per- 
formed by a two-dimensional two-tap filter including 1 
clock delay providers U, 12, 15, 16, multipliers 20, 21, 22, 

4Q 23, selector circuits 30, 31 and adder 40 in color separation 
circuit 8. More specifically, for the arrangements 1 and 4, 
selector 30 is switched to the side of terminal 30a by the 
switching signal, so that the output from multiplier 20 is 
selected, and selector 31 is switched to that terminal 31b to 

45 select the output from multiplier 23. The signal S33 which 
is the output from 1 clock delay provider 11 is multiplied by 
2 in multiplier 20, and input to adder 40 by selector 30. 
Meanwhile, the signal S22 which is the output signal from 
1 clock delay provider 16 is multiplied by 2 in multiplier 23, 

50 and input by selector 31 to adder 40. In adder 40, these 
inputs are added, and thus G signal at interpolation portion 
Y is generated. 

Meanwhile, for the arrangements 2 and 4, switching is 
controlled such that selector 30 is switched to terminal 30b 

55 to select the output from multiplier 22, and selector 31 is 
switched to terminal 31a to select the output from multiplier 
21. The signal S23 which is the output from 1 clock delay 
provider 12 is multiplied by 2 in multiplier 22 and input by 
selector 30, to adder 40. The signal S32 which is the output 

go signal from 1 clock delay provider 15 is multiplied by 2 in 
multiplier 21 and input by selector 31, to adder 40. In adder 
40, these inputs are added, whereby the G signal at the 
interpolation portion Y is generated. 
More specifically, in interpolation processing of G signal, 

65 of 4x4 pixels, only the image signals from central 2x2 pixels 
are used to generate the G signal. 
The procedure for generating R signal will be described. 



12/16/2003, EAST Version: 1.4.1 



5,852,468 

15 16 

When the R signal is to be generated at interpolation to adder 38. The signal value S42 is input through selectors 

portion Y, one of the four pixels surrounding interpolation 24 and 32 to adder 38. Therefore, a signal (S24+S42) is 

portion Y and two pixels of R color filters out of the output from adder 38. Finally, a signal (2xS22+S24+S42) 

outermost 12 pixels belonging to the same row or same would be output from adder 41, which is the R signal at the 

column as this one pixel are used, that is, R signals from the 5 interpolation portion Y 

total of three pixels are used. For the arrangement 4, selectors 26, 29, 32, and 33 are 

More specifically, for the arrangement 1, the weight switched to their a terminals, respectively, and selectors 24 

coefficient of "2" is allotted to the R signal from pixel P32 * nd 36 switched to their b terminals, respectively, 

as shown in FIG. IOC, and weight coefficient of "1" is ^nsequently, the signal value S23 is muttipbed by 2 in 

allotted to the R signals from pixels P12 and F34. R signals io ™ U £ ber 22 ' ™ d ^P^ trough selectors 29 and 36 to adder 

c a_ *». *i u- i- a u • 41. The signal value S21 is input to adder 38 through 

from these three pixels are multiplied by respective weight scJectors 33 ^ &{ j ^ S43 fe { throu ^ 

coefficients, and the multiplied values are added to provide sckctors 24 ^ 32 tQ addcf 3g XhcrefolCB a signal (S2 i + 

the R signal of the central portion Y S43) ^ output from adder 38 FinallVj a signal ( 2x S23+S21+ 

Similarly, for the arrangement 2, the weight coefficient of S43) would be output from adder 41, which will be the R 

"2" is allotted to R signal from pixel P33 as shown in FIG. 15 signal at interpolation portion Y. 

1C, and weight coefficient of "1" is allotted to the R signals The procedure for generating the B signal will be 

from pixels P13 and P31. The R signals from these three described in the following. 

pixels are multiplied by respective weight coefficients, and For generating the B signal at interpolation portion Y, one 

the multiplied values are added to provide the R signal of the of the four pixels surrounding the interpolation portion Y 

central portion For the arrangement 3, weight coefficient of 20 and two pixels out of outermost 12 pixels belonging to the 

"2" is allotted to R signal from pixel P22 and weight same row or same column as the one pixel, are used, that is, 

coefficient of "1*' is allotted to the R signals from pixels P24 B signals from the total of these three pixels are used. More 

and P42, as shown in FIG. 12C. The R signals from these specifically, for the arrangement 1, weight coefficient of "2" 

three pixels are multiplied by respective weight coefficients, ^ allotted to the B signal from pixel P23 and weight 

and the multiplied values are added to provide the R signal 25 coefficient of "1" is allotted to the B signals from pixels P21 

at the central portion Y anc * as snown m FIG. 10D. The B signals from these 

For the arrangement 4, weight coefficient of "2" is allotted *"* * pixe ^? r f. 7"? ,ied * ^P, ecti f 

. .t n • if- , ™* j • u . rc • „ and the multiphed values are added, whereby the B signal of 

to the R signal of pixel P23, and weight coefficient of 1 is tU „ • torn ™t: rt „ v ,v 

, . « ~\ „ f . - . , tCI. , , the interpolating portion Y is calculated. 

i £ C ^ 7 T ^ ™t ' S , 30 Similarly, for the arrangement 2, weight coefficient of "2" 

m % G v J I signals from these three pucels are fa alIottc / to mc B si | nal from ^ n2 aQd wei ^ t 

multiplied by respective weight coefficients, and the multi- coefficicnt of « r ^ aU otTed to the B signals from pixels P24 

phed values are added to provide the R signal at the central and p 42> as shown ^ nG UD ^ B signals from these 

portion Y mree pix^ are multiplied by respective weight coefficients 

In this manner, when R signal is to be calculated by 3S and the multiplied values are added, whereby the B signal at 

weighting and adding, a two-dimensional three-tap filter the interpolation portion Y is calculated, 
including all the 1 clock delay providers, multipliers 20, 21, For the arrangement 3, weight coefficient of "2" is allotted 

22 and 23, selector circuits 24, 25, 26, 27, 28, 29, 32, 33 and to the B signal from pixel P33 and weight coefficient of "1" 

36 and adders 38 and 41 of color separation circuit 8 is used. is allotted to the B signals from pixels P13 and P31 as shown 

More specifically, for the arrangement 1, selectors 27 and ^ in FIG. 12D. The B signals from these three pixels are 
36 are switched to their a terminals, respectively, and multiplied by respective weight coefficients, and the multi- 
selectors 25, 28, 32 and 33 are switched to their b terminals, phed values are added, whereby the B signal at the inter- 
respectively. Therefore, the signal value S32 is multiplied by polating portion Y is calculated. 

2 in multiplier 21, and input through selectors 28 and 36 to For the arrangement 4, weight coefficient of "2" is allotted 

adder 41. The signal value S12 is input through selectors 27 45 to the B signal from pixel P32 and weight coefficient of "1" 

and 33 to adder 38. The signal value S34 is input through is allotted to B signals from pixels P12 and P34, as shown 

selectors 25 and 32 to adder 38. Therefore, from adder 38, in FIG. 13D. The B signals from these three pixels are 

a signal (S12+S34) is output. Finally, from adder 41, a signal multiplied by respective weight coefficients, and by adding 

(2xS32+S12+S34) would be output, which is the R signal at these multiplied values, the B signal of the interpolating 

the interpolation portion Y 50 portion Y is calculated. 

For the arrangement 2, selectors 25, 28, and 36 are In this manner, when the B signal is to be calculated by 

switched to their a terminals, respectively, and selectors 27, weighting and adding, a two-dimensional three-tap filter 

32 and 33 are switched to their b terminals, respectively. including all the 1 clock delay providers, multipliers 20, 21, 

Consequently, the signal value S33 is multiplied by 2 in 22 and 23, selector circuits 24, 25, 26, 27, 28, 29, 34, 35, 37 

multiplier 20, and input to adder 41 through selectors 28 and 55 and adders 39 and 42 of color separation circuit 8 is used. 
36. The signal value S13 is input through selectors 27 and 33 More specifically, for the arrangement 1, selector circuits 

to adder 38. Signal value S31 is input to adder 38 through 26, 29, 34 and 35 are switched to their a terminals, 

selectors 25 and 32. Therefore, a signal (S13+S31) is output respectively, and selector circuits 24 and 37 are switched to 

from adder 38. Finally, a signal (2xS33+S13+S31) would be their b terminals, respectively. Therefore, the signal value 

output from adder 41, which is the R signal at interpolation 60 S23 is multiplied by 2 in multiplier 22, and input through 

portion Y selectors 29 and 37 to adder 42. The signal value S21 is input 

For the arrangement 3, selectors 24, 32 and 33 are through selectors 26 and 35 to adder 39. The signal value 

switched to their a terminals, respectively, and selectors 26, S43 is input through selectors 24 and 34 to adder 39. 

29 and 36 are switched to their b terminals, respectively. Therefore, a signal (S214S43) is output from adder 39. 

Consequently, the signal value S22 is multiplied by 2 in 65 Finally, a signal (2xS23+S21+S43) would be output from 

multiplier 23 and input through selectors 29 and 36 to adder adder 42, which will be the B signal at interpolation portion 

41. The signal value S24 is input through selectors 26 and 33 Y 
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For arrangement 2, selectors 24, 34, and 35 are switched 
to their a terminals, respectively, and selector circuits 26, 29 
and 37 are switched to their b terminals, respectively. 
Consequently, the signal value S22 is multiplied by 2 in 
multiplier 23 and input to adder 42 through selector circuits 
29 and 37. The signal value S24 is input through selector 
circuits 26 and 35 to adder 39. The signal value S42 is input 
through selector circuits 24 and 34 to adder 39. Therefore, 
a signal (S24+S42) is output from adder 39. Finally, a signal 
(2xS22+S24+S42) would be output from adder 42 which is 
the B signal at interpolation portion Y. 

For the arrangement 3, selector circuits 25, 28 and 37 are 
switched to their a terminals, respectively, and selecting 
circuits 27, 34 and 35 are switched to their b terminals, 
respectively. Therefore, the signal value S33 is multiplied by 
2 in multiplier 20 and input to adder 42 through selectors 28 
and 37. The signal value S13 is input through selector 
circuits 27 and 35 to adder 39. The signal value S31 is input 
through selector circuits 25 and 34 to adder 39. Therefore, 
a signal (S13+S31) is output from adder 39. Finally, a signal 
(2xS33+S13+S31) would be output from adder 42, which is 
the B signal at interpolation portion Y. 

For the arrangement 4, selector circuits 27 and 37 are 
switched to their a terminals, respectively, and selector 
circuits 25, 28, 34 and 35 are switched to their b terminals, 
respectively. Consequently, the signal value S32 is multi- 
plied by 2 in multiplier 21, and input to adder 42 through 
selector circuits 28 and 37. The signal value S34 is input 
through selector circuits 25 and 34 to adder 39. Therefore, 
a signal (S12+S34) is output from adder 39. Finally, from 
adder 42, a signal (2xS32+S12+S34) would be output, 
which is the B signal at interpolating portion Y 

In the interpolation of R, G and B signals described above, 
it is also possible to utilize a weighted mean in which color 
signal values for each arrangement are divided by 4, i.e., the 
sum of weight coefficients. 

In this manner, regardless of which of the four different 
arrangements corresponds to the arrangement of an arbitrary 
4x4 pixel block on CCD 1, three primary signals of R, G and 
B can be calculated at interpolation portion Y of the pixel 
block in color separation circuit 8. 

When generation of color signals at interpolation portion 
Y of a certain 4x4 pixel block is completed and image signal 
from the next pixel is output from A/D converter 4, the 
object pixel block is shifted by one pixel in horizontal 
direction, and similar processing is repeated. When this 
horizontal movement results in complete movement of one 
row in CCD 1, the pixel block of which interpolation is to 
be performed returns to the initial positioD horizontally 
while it is shifted by one pixel in vertical direction. 

As the pixel block is shifted, the interpolation portion Y 
is also shifted in horizontal and vertical directions succes- 
sively. Finally, as shown in FIG. 14, R, G and B signals at 
an intersection of four pixels on the CCD 1 are calculated. 
Here, the interpolation portion marked by the hatching in 
FIG. 14 indicates a central point of any pixel block if a 4x4 
pixel block can be actually formed on the CCD 1. 

Other than the hatched portion, it is possible to calculate 
values corresponding to R, G and B signals, based on the 
signals output from CCD 1 successively from color sepa- 
ration circuit 8. However, a 4x4 pixel block cannot be 
physically set on the CCD 1, and hence the calculated value 
is meaningless as data for interpolation. 

Generally, on CCD 1, non-effective pixels which are not 
displayed on a monitor are arranged on left, right, upper and 
lower edges. Therefore, if the interpolation portions denoted 



52,468 

18 

by the hatched portions in FIG. 14 are used as effective 
pixels and portions not hatched are set as non-effective 
pixels, only the color signals obtained from the effective 
pixels would be visible signals. 

5 In FIG. 14, the pixels of CCD 1 have been described as 
having a scale of 8x8 for convenience of description. 
Therefore, hatched portions are smaller as compared with 
portions not hatched. However, generally, the total number 
of pixels of a CCD 1 is as large as about 530x500, and most 

10 of the total number of pixels can be occupied by the hatched 
portions, that is, portions where 4x4 pixel blocks can be set. 
In other words, most pixels can be used as effective pixels. 

The transfer function of a two-dimension non-recursive 
digital filter for performing interpolation of color signals in 

15 color separation circuit 8 in the above described manner are 
as follows. 

[ARRANGEMENT 1] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G 
2Q SIGNAL) 

H (z)-2hmz- m =lxz-°+lxz=- 1 =l+ z - 1 

(HORIZONTAL DIRECTION OF R SIGNAL) H (z)= 

Zhmz- m -0xz-°+3xz- 1 +0x0- 2 +lx2- 3 -3z- J +z- 3 
(VERTICAL DIRECTION OF R SIGNAL) 
25 H (z)=2hmz- m =lxz-°+0xz- 1 3xz- 2 40xz- 3 =l+3z- 2 

(HORIZONTAL DIRECTION OF B SIGNAL H (z)- 

Zhmz" m =lxz~°+0xz~ 1 +3xz- 2 0xz- 3 «l+3z~ 2 
(VERTICAL DIRECTION OF B SIGNAL) H (z)=2hmz- 
«=0xz"°+3xz~ 1 +0xz" 2 +lxz" 3 =3z" 1 +z" 3 
30 [ARRANGEMENT 2] 

(HORIZONTAL & VERTICAL DIRECTIONS OF G 

SIGNAL) 
H (z)=£hmz-'"-lxz- 0 +lxz- 1 =l+z- 1 
35 (HORIZONTAL DIRECTION OF R SIGNAL) 
H (z)=Xhmz- w, =lxz-°+0xz- 1 «3xz- 2 +0xz- 3 =l+3z- 2 
(VERTICAL DIRECTION OF R SIGNAL) 
H (z)=2hmz-'"-lxz- o +0xz- 1 +3xz- 2 +0xz- 3 =l+3z- 2 
(HORIZONTAL DIRECTION OF B SIGNAL) 
40 H (z)=»2hmz-' n -0xz- o +3xz- 1 +0xz- 2 +lxz- 3 3z- 1 +z- 3 

(VERTICAL DIRECTION OF B SIGNAL) H (z)-Zhmz" 
m-0xz"°3xz~ 1 +0xz" 2 +lxz" 3 -3z" :i +z" 3 
[ARRANGEMENT 3] 
45 (HORIZONTAL & VERTICAL DIRECTIONS OF G 
SIGNAL) 
H (z)-2hmz-' n -lxz-- 0 +lxz-M+z- 1 
H (z)-2hmz-" , -0xz-°+3xz- 1 +Oxz- 2 +lxz- 3 -3z- 1 +z- 3 
(VERTICAL DIRECTION OF R SIGNAL) 
50 H (z)«2hmz-" , =0xz-°+3xz- 1 +Oxz- 2 +lxz- 3 »3z- 1 +z- 3 
(HORIZONTAL DIRECTION OF B SIGNAL) 
H (z)-2hms- m «lxz-°+0xz-+3xz- 2 +0xz- 3 -l+3z- 2 
(VERTICAL DIRECTION OF B SIGNAL) H (z)Zhms- 
m=lxz~ o 40xz -1 +3xz~ 2 +0xz"M+3z~ 2 
55 [ARRANGEMENT 4] 

(HORIZONTAL & VERTICAL SIGNALS OF G 

SIGNAL) 
H (z^Zhmz-^lxz-Mxz-M+z- 1 
(HORIZONTAL DIRECTION OF R SIGNAL) H (z)= 

2hmz- m =lxz-°+0xz- 1 +3xz- 2 +0xz- 3 -l+3z- 2 
VERTICAL DIRECTION OF R SIGNAL) H (z)-Zhmz" 

m-0xz~°+3xz~ 1 +0xz" 2 +lxz" 3 -3z" 1 +z" 3 
(HORIZONTAL DIRECTION OF B SIGNAL) 
65 H (z)=Zhmz- m =0xz J °+3xz- 1 +0xz- 2 +lxz- 3 »3z- 1 +z- 3 
(VERTICAL DIRECTION OF B SIGNAL) 
H (z)=»2hmz-m=»lxz-°+0xz- 1 +3xz" 2 +0x Z - 3 =l+3z" 2 
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The frequency characteristics of color signals which are 
the output signals of the two-dimensional non-recursive 
digital filter having such transfer functions are as shown in 
FIG. 15. In the graph, the curve Ql shows horizontal and 
vertical characteristics of G signal, and curve Q2 shows 5 
horizontal and vertical characteristics of R and B signals. 

As is apparent from FIGS. 10B to 14B, the pixels used for 
generating the G signal exist at the central two columns out 
of four columns when viewed horizontally, and exist in the 
central two columns when viewed vertically. Therefore, in 
any of the arrangements 1 to 4, the gain lowers near the 
Nyquist frequency. 

The sum of the weight coefficients of the pixels used for 
generating the R signal, when viewed horizontal, any of the 
central two columns out of four columns is "3", and in the 
column outer by one column, it would be "1". Meanwhile, 1S 
when viewed vertically, either of the central two rows out of 
four rows is always "3", and an outer row spaced by one row 
is "1". Therefore, frequency characteristic of R signal is 
always as represented by Q2. 

Similarly to R signals, the sum of the weight coefficients 20 
of the pixels used for generating the B signal is, when 
viewed horizontally, either of the central two of four col- 
umns is always "3" and an outer column spaced by one 
column is "1". When viewed vertically, it is always "3" in 
either of the central two rows out of four rows, and in the 25 
outer row spaced by one row, it is "1". Therefore, the 
frequency characteristic of the B signal is always as repre- 
sented by Q2. 

As is apparent from FIG. 15, the frequency characteristic 
Ql of the G signal, which has highest contribution to 30 
brightness among then three primary signals, exhibits rela- 
tively little attenuation in the higher frequency range as 
compared with the curve P2 shown in FIG. 7, which means 
that higher resolution is possible. 

Further, at l A Nyquist frequency, the difference between 35 
frequency characteristic of R and B signals over the fre- 
quency characteristic of G signal is considerably smaller 
than in FIG. 7, and hence generation of a false color signal 
can be suppressed. 

Therefore, according to the present invention, a frequency ^ 
characteristic which exhibit relatively little attenuation in 
high frequency range can be provided for the G signal which 
has highest contribution to brightness, so that high resolu- 
tion can be obtained. Further, at ¥i Nyquist frequency, the 
difference between frequency characteristic of R and B 4S 
signals and G signal is small, and hence generation of a false 
color signal can be suppressed. Further, the same frequency 
characteristic can be obtained for every pixel, for each of the 
three primary colors. 

Although the present invention has been described and 50 
illustrated in detail, it is clearly understood that the same is 
by way of illustration and example only and is not to be 
taken by way of limitation, the spirit and scope of the present 
invention being limited only by the terms of the appended 
claims. S5 

What is claimed is: 

1. A 1-chip color video camera, comprising: 

solid-state image sensing means provided with photoelec- 
tric converting elements corresponding to respective 
pixels arranged in an array; 60 

said solid-state image sensing means including on its light 
receiving surface a color filter array in which primary 
color filters are placed in a prescribed arrangement 
corresponding to said photoelectric converting ele- 
ments; and 65 

interpolating means, receiving an output from said solid- 
state image sensing means, for outputting color signals; 



said interpolating means including: 
parallel color signal output means, receiving said out- 
put from said solid-state image sensing means, for 
outputting, as said color signals, color signals cor- 
responding to a prescribed even number of rows of 
said pixels in synchronization with and parallel to 
each other successively, column by column; 
control means for outputting a synchronized interpolation 
designating signal in accordance with correspondence 
between said color signals, output column by column 
from said parallel color signal output means, and an 
arrangement of predetermined weight coefficients for 
the prescribed arrangement of said color filter array; 
and 

color separation means, receiving said outputs from said 
parallel color signal output means, for interpolating 
color signals from a pixel block including pixels of said 
prescribed even number of rows and a prescribed even 
number of columns in accordance with said interpola- 
tion designating signal, and for successively outputting 
color signals in synchronization with and correspond- 
ing to a central position of said pixel block. 

2. A 1-chip color video camera, comprising: 
solid-state image sensing means provided with photoelec- 
tric converting elements corresponding to respective 
pixels arranged in an array; 

said solid-state image sensing means including on its light 
receiving surface a color filter array in which primary 
color filters are arranged mosaic-wise corresponding to 
said photoelectric converting elements; 

said color filter array having green filters placed in a 
diagonal direction in an arbitrary color filter arrange- 
ment of two rows by two columns; and 

interpolating means, receiving an output from said solid- 
state image sensing means, for interpolating a green 
signal component at a central portion of a pixel block 
corresponding to said arbitrary color filter arrangement 
of two rows by two columns, using a mean value of a 
plurality of green signals from those of said photoelec- 
tric converting elements which correspond to said 
green filters placed in the diagonal direction. 

3. A 1-chip color video camera, comprising: 
solid-state image sensing means provided with photoelec- 
tric converting elements corresponding to respective 
pixels arranged in an array; 

said solid-state image sensing means including, on its 
light receiving surface, a color filter array in which 
primary color filters are arranged mosaic-wise corre- 
sponding to said photoelectric converting elements; 
and 

said color filter array having red filters and blue filters 
arranged checkerwise, with rows including said red 
filters and rows including said blue filters being 
arranged alternately; and 

interpolating means for interpolating, based on signals 
from a pixel block including sixteen pixels of four rows 
by four columns, a color signal component of each of 
red and blue at a central portion of said pixel block; 

said interpolating means including a two-dimensional 
non-recursive digital filter circuit for switching, in 
accordance with an arrangement of the color filters 
corresponding to said sixteen pixels, values of sums of 
weight coefficients of every column in a vertical direc- 
tion to [0, 3, 0, 1] or [1, 0, 3, 0]. 



12/16/2003, EAST Version: 1.4.1 



5,852 : 

21 

4. A 1-chip color video camera comprising: 
solid-state image sensing means provided with photoelec- 
tric converting elements corresponding to respective 
pixels arranged in an array; 

said solid-state image sensing means including, on its 5 
light receiving surface, a color filter array in which 
primary color filters are arranged mosaic-wise corre- 
sponding to said photoelectric converting elements; 
and 

said color filter array having red filters and blue filters 10 
arranged checkerwise, with rows including said red 
filters and rows including said blue filters being 
arranged alternately; and 

interpolating means for interpolating, based on signals 15 
from a pixel block including sixteen pixels of four rows 
by four columns, a color signal component of each of 
red and blue at a central portion of said pixel block; 

said interpolating means including a two-dimensional 
non-recursive digital filter for switching, in accordance 2 o 
with an arrangement of the color filters corresponding 
to said sixteen pixels, values of sums of weight coef- 
ficients of every row in a horizontal direction to [0, 3, 
0, 1] or [1, 0, 3, 0]. 

5. A 1-chip color video camera, comprising: 25 
solid-state image sensing means provided with photoelec- 
tric converting elements corresponding to respective 
pixels arranged in an array; 

said solid-state image sensing means including, on its 
light receiving surface, a color filter array in which 30 
primary color filters are arranged mosaic-wise corre- 
sponding to said photoelectric converting elements; 
and 

said color filter array having red filters and blue filters 
arranged checkerwise, with rows including said red 35 
filters and rows including said blue filters being 
arranged alternately; and 

interpolating means for interpolating, based on signals 
from a pixel block including sixteen pixels of four rows 
by four columns, a color signal component of each of 40 
red and blue at a central portion of said pixel block; and 

said interpolating means including a two-dimensional 
non-recursive digital filter circuit; 

wherein when said pixel block of sixteen pixels includes 45 
sixteen pixels of four rows by four columns from (m, 
n) to (m+3, n+3) with m and n being integers; and 

said two-dimensional non-recursive digital filter circuit 
performs interpolation by: 

setting a weight coefficient of a pixel (m+1, n+1) to 2 50 
and a weight coefficient of each of pixels (m+1, n30 
3) and (m+3, n+1) to 1, when either a red or a blue 
color filter is placed on the pixel (m+1, n+1); 
setting a weight coefficient of a pixel (m+1, n+2) to 2 
and a weight coefficient of each of pixels (m+1, n) 55 
and (m+3, n+2) to 1, when either a red or a blue color 
filter is placed on the pixel (m+1, n+2); 
setting a weight coefficient of a pixel (m+2, n+1) to 2 
and a weight coefficient of each of pixels (m, n+1) 
and (m+2, n30 3) to 1, when either a red or a blue 6 o 
color filter is placed on the pixel (m+2, n+1); and 
setting a weight coefficient of a pixel (m+2, n+2) to 2 and 
a weight coefficient of each of pixels (m, n+2) and 
(m+2, n) to 1, when either a red or a blue color filter is 
placed on the pixel (m+2, n+2). 65 

6. The 1-chip color video camera according to claim 1, 
further comprising: 
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analog-to-digital converting means, receiving said output 
from said solid-state image sensing means, for output- 
ting a corresponding digital signal thereto; 

wherein 

said parallel color signal output means includes (2k-l) 
line memories, where 2k represents said prescribed 
even number; 

each of said line memories having transfer capacity 
corresponding to a row of said pixels, and said line 
memories being connected in series to receive as an 
initial stage input, an output from said analog-to-digital 
converting means; and 
said parallel color signal output means outputs, in parallel, 
said output from said analog-to-digital converting 
means and outputs from said fine memories; and 
said color separation means includes: 

a two-dimensional transfer register, receiving said par- 
allel color signal outputs, for successively transfer- 
ring said parallel color signal outputs to a prescribed 
direction, and holding at most 2kx2k color signal 
values; and 

interpolation calculation means, responsive to said 
interpolation designating signal, for adding corre- 
sponding ones of said color signals held in said 
two-dimensional transfer register to each other, in 
accordance with an arrangement of said weight coef- 
ficients. 

7. The 1-chip color video camera according to claim 6, 
wherein: 

said color filter array includes green filters arranged in a 
diagonal direction in an arbitrary color filter arrange- 
ment of two rows by two columns; and 

said interpolation calculation means outputs, in accor- 
dance with said interpolation designating signal, a 
green signal corresponding to a central position of said 
pixel block as either a mean value of color signal values 
of (k, k) and (k+1, k+1) or a mean value of color signal 
values of (k+1, k) and (k, k+1), of the two-dimensional 
arrangement of said 2kx2k color signal values. 

8. The 1-chip color video camera according to claim 6, 
wherein: 

said prescribed even number is 4; 

said color filter array includes red filters and blue filters 
arranged checkerwise, with rows including said red 
filters and rows including said blue filters being 
arranged alternately; 

said two-dimensional transfer register holds signals from 
a pixel block including sixteen pixels of four rows by 
four columns; and 

said interpolation calculation means switches, in accor- 
dance with said interpolation designating signal, values 
of sums of weight coefficients in every column in a 
vertical direction for color signals corresponding to 
said sixteen pixels to [0, 3, 0, 1] or [1, 0, 3, 0], for 
interpolating and outputting color signal component of 
each of red and blue at a central portion of said pixel 
block. 

9. The 1-chip color video camera according to claim 6, 
wherein: 

said prescribed even number is 4; 

said color filter array includes red filters and blue filters 

arranged checkerwise, with rows including said red 

filters and rows including said blue filters being 

arranged alternately; 
said two-dimensional transfer register holds signals from 

a pixel block including sixteen pixels of four rows by 

four columns; and 
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said interpolation calculation means for switching, in 
accordance with said interpolation designating signal, 
values of sums of weight coefficients of every row in 
horizontal direction for color signals corresponding to 
said sixteen pixels to [0, 3, 0, 1] or [1, 0, 3, 0], and for 
interpolating and output ting a color signal component 
of each of red and blue at a central portion of said pixel 
block. 

10. The 1-chip color video camera according to claim 6, 
wherein: 

said prescribed even number is 4; 

said color filter array includes red filters and blue filters 
arranged checkerwise, with rows including said red 
filters and rows including said blue filters being 
arranged alternately; 

said two-dimensional transfer register holds signals from 
a pixel block including sixteen pixels of four rows by 
four columns; and 

said interpolation calculation means performs interpola- 
tion in accordance with said interpolation designating 
signal, where said pixel block of 16 pixels includes 
sixteen pixels of four rows by four columns of (m, n) 
to (m+3, n30 3), by: 

setting a weight coefficient of a pixel (m+1, n+1) to 2 
and a weight coefficient of each of pixels (m+1, n30 
3) and (m+3, n+1) to 1, when either a red or a blue 
color filter is placed on the pixel (m+1, n+1); 

setting a weight coefficient of a pixel (m+1, n+2) to 2 
and a weight coefficient of each of pixels (m+1, n) 
and (m+3, n+2) to 1, when either a red or a blue color 
filter is placed on the pixel (m+1, n+2); 

setting a weight coefficient of a pixel (m+2, n+1) to 2 
and a weight coefficient of each of pixels (m, n+1) 
and (n+2, n30 3) to 1, when either a red or a blue 35 
color filter is placed on the pixel (m+2, n+1); and 

setting a weight coefficient of a pixel (m+2, n+2) to 2 
and a weight coefficient of each of pixels (m, n+2) 
and (m+2, n) to 1, when either a red or a blue color 
filter is placed on the pixel (m+2, n+2). 40 

11. A 1-chip color video camera, comprising: 
solid-stale image sensing means provided with photoelec- 
tric converting elements corresponding to respective 
pixels arranged in an array; 

said solid-state image sensing means, including on its 45 
light receiving surface a color filter array in which 
primary color filters are arranged mosaic-wise corre- 
sponding to said photoelectric converting elements; 
and 

interpolating means for generating each of a plurality of 50 
primary color signal components at a position shifted 
by half a pixel in horizontal and vertical directions from 
a center of an arbitrary pixel, from color signal com- 
ponents of a plurality of neighboring pixels, based on 
a predetermined weighted average of corresponding 55 
primary color signal components from a plurality of 
neighboring pixels to the center pixel and according to 
a prescribed arrangement of said color filter array. 

12. A 1-chip color video camera, comprising: 
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solid-state image sensing means having color filters of 
three primary colors of R, G and B (red, green and blue, 
respectively) arranged mosaic-wise corresponding to 
respective pixels; 

color separation means for processing a signal from said 
solid-state image sensing means; 

said color separation means including signal processing 
means for performing weighted average processing to 
obtain a weighted average of R, G and B color signal 
components at a central position of a pixel block 
consisting of 16 pixels of 4 rows by 4 columns of (m, 
n) to (m+3, n30 3), where m and n are integers, using 
a prescribed weight coefficient for a signal from a 
prescribed pixel in said pixel block, wherein: 
if a first value is larger than a second value, said 
weighted average processing is performed when: 

i) the R or B color filter is arranged on a pixel at 
(m+1, n+1), and using a weight coefficient for the 
pixel (m+1, n+1) as the first value and a weight 
coefficient of pixels at (m+1, n30 3) and (m+3, 
n+1) as the second value; 

ii) the R or B color filter is arranged on a pixel at 
(m+1, n+2), and using a weight coefficient for the 
pixel (m+1, n+2) as the first value and a weight 
coefficient of pixels at (m+1, n) and (m+3, n+2) as 
the second value; 

iii) the R or B color filter is arranged on a pixel at 
(m+2, n+1), and using a weight coefficient for the 
pixel (m+2, n+1) as the first value and a weight 
coefficient of pixels at (m, n+1) and (m+2, n30 3) 
as the second value; and 

iv) the R or B color filter is arranged on a pixel at 
(m+2, n+2), and using a weight coefficient for the 
pixel (m+2, n+2) as the first value and a weight 
coefficient of pixels at (m, n+2) and (m+2, n) as 
the second value. 

13. A 1-chip video camera, comprising: 

solid-state image sensing means having color filters of 
three primary colors R, G and B (red, green and blue, 
respectively) arranged mosaic-wise corresponding to 
respective pixels; and 

interpolating means for generating R, G and B color 
signal data at a central position of a pixel block, the 
block consisting of an arbitrary v rows by v columns, 
where v is an even number, and using a prescribed 
weight coefficient for a signal for each color for a 
prescribed pixel in said pixel block. 

14. A 1-chip color video camera according to claim 13, 
wherein said color filters comprises: 

B color filters and G color filters arranged alternately for 
each of the pixels in one row of every two rows 
adjacent to each other in a vertical direction; and 

G filters and R filters arranged alternately offset by one 
pixel in the vertical direction from said one row, and on 
each one of the pixels of the other one of said two rows. 



12/16/2003, EAST Version: 1.4.1 



